Automatic classification of a taxon-rich community recorded in the wild

Effects of a large-scale bioretention installation on the species composition of an urban bird community as determined by passive acoustic monitoring

As urbanization accelerates worldwide, municipalities are attempting to construct new green spaces within their borders. The perceived ecological value of these places is frequently tied to their ability to attract urban wildlife, such as birds, which can easily be observed and enjoyed. As one strategy, stormwater is now frequently managed with green infrastructure: planted areas that retain and treat stormwater rather than merely directing it to surface waters. While these practices have the potential to provide habitat for urban wildlife, the ecological effects of these systems are largely unknown. To assess whether one green infrastructure project increases habitat value, we used passive acoustic monitoring to survey urban bird communities in and near a large green infrastructure project in Columbus, Ohio (USA). Bird communities near bioretention cells (rain gardens) were compared to those at nearby lawns and remnant or restored natural areas. We found that recently installed bioretention cells tended to support more omnivores, lower-canopy foraging species, and species from a higher diversity of feeding guilds than did nearby lawn control sites. We were unable to detect effects of nearby bioretention installations on bird species richness at other sites. The observed differences in species richness were fairly small, and we urge caution when anticipating the habitat value of bioretention cells, at least for bird species. However, the results that we observed suggest that bioretention cells could have a more positive impact on bird communities in different contexts or using different design strategies. The bioretention cells surveyed in this study were small and only planted in grasses and forbs, potentially limiting their ability to offer complex habitat. They were also relatively young, and future work is needed to determine their long-term effect on avian communities and biodiversity of other taxa.

Analytical approaches for evaluating passive acoustic monitoring data: A case study of avian vocalizations

The interface between field biology and technology is energizing the collection of vast quantities of environmental data. Passive acoustic monitoring, the use of unattended recording devices to capture environmental sound, is an example where technological advances have facilitated an influx of data that routinely exceeds the capacity for analysis. Computational advances, particularly the integration of machine learning approaches, will support data extraction efforts. However, the analysis and interpretation of these data will require parallel growth in conceptual and technical approaches for data analysis. Here, we use a large hand‐annotated dataset to showcase analysis approaches that will become increasingly useful as datasets grow and data extraction can be partially automated.

We propose and demonstrate seven technical approaches for analyzing bioacoustic data. These include the following: (1) generating species lists and descriptions of vocal variation, (2) assessing how abiotic factors (e.g., rain and wind) impact vocalization rates, (3) testing for differences in community vocalization activity across sites and habitat types, (4) quantifying the phenology of vocal activity, (5) testing for spatiotemporal correlations in vocalizations within species, (6) among species, and (7) using rarefaction analysis to quantify diversity and optimize bioacoustic sampling.

To demonstrate these approaches, we sampled in 2016 and 2018 and used hand annotations of 129,866 bird vocalizations from two forests in New Hampshire, USA, including sites in the Hubbard Brook Experiment Forest where bioacoustic data could be integrated with more than 50 years of observer‐based avian studies. Acoustic monitoring revealed differences in community patterns in vocalization activity between forests of different ages, as well as between nearby similar watersheds. Of numerous environmental variables that were evaluated, background noise was most clearly related to vocalization rates. The songbird community included one cluster of species where vocalization rates declined as ambient noise increased and another cluster where vocalization rates declined over the nesting season. In some common species, the number of vocalizations produced per day was correlated at scales of up to 15 km. Rarefaction analyses showed that adding sampling sites increased species detections more than adding sampling days.

Although our analyses used hand‐annotated data, the methods will extend readily to large‐scale automated detection of vocalization events. Such data are likely to become increasingly available as autonomous recording units become more advanced, affordable, and power efficient. Passive acoustic monitoring with human or automated identification at the species level offers growing potential to complement observer‐based studies of avian ecology.

Diversity Monitoring of Coexisting Birds in Urban Forests by Integrating Spectrograms and Object-Based Image Analysis

In the context of rapid urbanization, urban foresters are actively seeking management monitoring programs that address the challenges of urban biodiversity loss. Passive acoustic monitoring (PAM) has attracted attention because it allows for the collection of data passively, objectively, and continuously across large areas and for extended periods. However, it continues to be a difficult subject due to the massive amount of information that audio recordings contain. Most existing automated analysis methods have limitations in their application in urban areas, with unclear ecological relevance and efficacy. To better support urban forest biodiversity monitoring, we present a novel methodology for automatically extracting bird vocalizations from spectrograms of field audio recordings, integrating object-based classification. We applied this approach to acoustic data from an urban forest in Beijing and achieved an accuracy of 93.55% (±4.78%) in vocalization recognition while requiring less than ⅛ of the time needed for traditional inspection. The difference in efficiency would become more significant as the data size increases because object-based classification allows for batch processing of spectrograms. Using the extracted vocalizations, a series of acoustic and morphological features of bird-vocalization syllables (syllable feature metrics, SFMs) could be calculated to better quantify acoustic events and describe the soundscape. A significant correlation between the SFMs and biodiversity indices was found, with 57% of the variance in species richness, 41% in Shannon’s diversity index and 38% in Simpson’s diversity index being explained by SFMs. Therefore, our proposed method provides an effective complementary tool to existing automated methods for long-term urban forest biodiversity monitoring and conservation.

Sounding the Call for a Global Library of Underwater Biological Sounds

Aquatic environments encompass the world’s most extensive habitats, rich with sounds produced by a diversity of animals. Passive acoustic monitoring (PAM) is an increasingly accessible remote sensing technology that uses hydrophones to listen to the underwater world and represents an unprecedented, non-invasive method to monitor underwater environments. This information can assist in the delineation of biologically important areas via detection of sound-producing species or characterization of ecosystem type and condition, inferred from the acoustic properties of the local soundscape. At a time when worldwide biodiversity is in significant decline and underwater soundscapes are being altered as a result of anthropogenic impacts, there is a need to document, quantify, and understand biotic sound sources–potentially before they disappear. A significant step toward these goals is the development of a web-based, open-access platform that provides: (1) a reference library of known and unknown biological sound sources (by integrating and expanding existing libraries around the world); (2) a data repository portal for annotated and unannotated audio recordings of single sources and of soundscapes; (3) a training platform for artificial intelligence algorithms for signal detection and classification; and (4) a citizen science-based application for public users. Although individually, these resources are often met on regional and taxa-specific scales, many are not sustained and, collectively, an enduring global database with an integrated platform has not been realized. We discuss the benefits such a program can provide, previous calls for global data-sharing and reference libraries, and the challenges that need to be overcome to bring together bio- and ecoacousticians, bioinformaticians, propagation experts, web engineers, and signal processing specialists (e.g., artificial intelligence) with the necessary support and funding to build a sustainable and scalable platform that could address the needs of all contributors and stakeholders into the future.

Evaluation of acoustic pattern recognition of nightingale (Luscinia megarhynchos) recordings by citizens

Acoustic pattern recognition methods introduce new perspectives for species identification, biodiversity monitoring and data validation in citizen science but are rarely evaluated in real world scenarios. In this case study we analysed the performance of a machine learning algorithm for automated bird identification to reliably identify common nightingales (Luscinia megarhynchos) in field recordings taken by users of the smartphone app Naturblick. We found that the performance of the automated identification tool was overall robust in our selected recordings. Although most of the recordings had a relatively low confidence score, a large proportion of the recordings were identified correctly.

Speeding up training of automated bird recognizers by data reduction of audio features

Automated acoustic recognition of birds is considered an important technology in support of biodiversity monitoring and biodiversity conservation activities. These activities require processing large amounts of soundscape recordings. Typically, recordings are transformed to a number of acoustic features, and a machine learning method is used to build models and recognize the sound events of interest. The main problem is the scalability of data processing, either for developing models or for processing recordings made over long time periods. In those cases, the processing time and resources required might become prohibitive for the average user. To address this problem, we evaluated the applicability of three data reduction methods. These methods were applied to a series of acoustic feature vectors as an additional postprocessing step, which aims to reduce the computational demand during training. The experimental results obtained using Mel-frequency cepstral coefficients (MFCCs) and hidden Markov models (HMMs) support the finding that a reduction in training data by a factor of 10 does not significantly affect the recognition performance.

Automatic Classification of Cat VocalizationsEmitted in Different Contexts

Simple Summary

Cat vocalizations are their basic means of communication. They are particularly important in assessing their welfare status since they are indicative of information associated with the environment they were produced, the animal’s emotional state, etc. As such, this work proposes a fully automatic framework with the ability to process such vocalizations and reveal the context in which they were produced. To this end, we used suitable audio signal processing and pattern recognition algorithms. We recorded vocalizations from Maine Coon and European Shorthair breeds emitted in three different contexts, namely waiting for food, isolation in unfamiliar environment, and brushing. The obtained results are excellent, rendering the proposed framework particularly useful towards a better understanding of the acoustic communication between humans and cats.

Abstract

Cats employ vocalizations for communicating information, thus their sounds can carry a wide range of meanings. Concerning vocalization, an aspect of increasing relevance directly connected with the welfare of such animals is its emotional interpretation and the recognition of the production context. To this end, this work presents a proof of concept facilitating the automatic analysis of cat vocalizations based on signal processing and pattern recognition techniques, aimed at demonstrating if the emission context can be identified by meowing vocalizations, even if recorded in sub-optimal conditions. We rely on a dataset including vocalizations of Maine Coon and European Shorthair breeds emitted in three different contexts: waiting for food, isolation in unfamiliar environment, and brushing. Towards capturing the emission context, we extract two sets of acoustic parameters, i.e., mel-frequency cepstral coefficients and temporal modulation features. Subsequently, these are modeled using a classification scheme based on a directed acyclic graph dividing the problem space. The experiments we conducted demonstrate the superiority of such a scheme over a series of generative and discriminative classification solutions. These results open up new perspectives for deepening our knowledge of acoustic communication between humans and cats and, in general, between humans and animals.

Animal Sound Identifier (ASI): software for automated identification of vocal animals

Automated audio recording offers a powerful tool for acoustic monitoring schemes of bird, bat, frog and other vocal organisms, but the lack of automated species identification methods has made it difficult to fully utilise such data. We developed Animal Sound Identifier (ASI), a MATLAB software that performs probabilistic classification of species occurrences from field recordings. Unlike most previous approaches, ASI locates training data directly from the field recordings and thus avoids the need of pre-defined reference libraries. We apply ASI to a case study on Amazonian birds, in which we classify the vocalisations of 14 species in 194 504 one-minute audio segments using in total two weeks of expert time to construct, parameterise, and validate the classification models. We compare the classification performance of ASI (with training templates extracted automatically from field data) to that of monitoR (with training templates extracted manually from the Xeno-Canto database), the results showing ASI to have substantially higher recall and precision rates.

Vocalizations of adult male Asian koels (Eudynamys scolopacea) in the breeding season

Defining the vocal repertoire provides a basis for understanding the role of acoustic signals in sexual and social interactions of an animal. The Asian koel (Eudynamys scolopacea) is a migratory bird which spends its summer breeding season in the plains of Pakistan. The bird is typically wary and secretive but produces loud and distinct calls, making it easily detected when unseen. Like the other birds in the wild, presumably Asian koels use their calls for social cohesion and coordination of different behaviors. To date, the description of vocal repertoire of the male Asian koel has been lacking. Presently we analyzed and described for the first time the vocalizations of the adult male Asian koel, recorded in two consecutive breeding seasons. Using 10 call parameters, we categorized the vocalization type into six different categories on the basis of spectrogram and statistical analyses, namely the; “type 1 cooee call”, “type 2 cooee call”, “type 1 coegh call”, “type 2 coegh call”, “wurroo call” and “coe call”. These names were assigned not on the basis of functional analysis and were therefore onomatopoeic. Stepwise cross validated discriminant function analysis classified the vocalization correctly (100%) into the predicted vocal categories that we initially classified on the basis of spectrographic examination. Our findings enrich the biological knowledge about vocalizations of the adult male Asian koel and provide a foundation for future acoustic monitoring of the species, as well as for comparative studies with vocalizations of other bird species of the cuckoo family. Further studies on the vocalizations of the Asian koel are required to unravel their functions in sexual selection and individual recognition.

PROTAX-Sound: A probabilistic framework for automated animal sound identification

Autonomous audio recording is stimulating new field in bioacoustics, with a great promise for conducting cost-effective species surveys. One major current challenge is the lack of reliable classifiers capable of multi-species identification. We present PROTAX-Sound, a statistical framework to perform probabilistic classification of animal sounds. PROTAX-Sound is based on a multinomial regression model, and it can utilize as predictors any kind of sound features or classifications produced by other existing algorithms. PROTAX-Sound combines audio and image processing techniques to scan environmental audio files. It identifies regions of interest (a segment of the audio file that contains a vocalization to be classified), extracts acoustic features from them and compares with samples in a reference database. The output of PROTAX-Sound is the probabilistic classification of each vocalization, including the possibility that it represents species not present in the reference database. We demonstrate the performance of PROTAX-Sound by classifying audio from a species-rich case study of tropical birds. The best performing classifier achieved 68% classification accuracy for 200 bird species. PROTAX-Sound improves the classification power of current techniques by combining information from multiple classifiers in a manner that yields calibrated classification probabilities.

An FPGA-Based WASN for Remote Real-Time Monitoring of Endangered Species: A Case Study on the Birdsong Recognition of Botaurus stellaris

Fast environmental variations due to climate change can cause mass decline or even extinctions of species, having a dramatic impact on the future of biodiversity. During the last decade, different approaches have been proposed to track and monitor endangered species, generally based on costly semi-automatic systems that require human supervision adding limitations in coverage and time. However, the recent emergence of Wireless Acoustic Sensor Networks (WASN) has allowed non-intrusive remote monitoring of endangered species in real time through the automatic identification of the sound they emit. In this work, an FPGA-based WASN centralized architecture is proposed and validated on a simulated operation environment. The feasibility of the architecture is evaluated in a case study designed to detect the threatened Botaurus stellaris among other 19 cohabiting birds species in The Parc Natural dels Aiguamolls de l’Empordà, showing an averaged recognition accuracy of 91% over 2h 55’ of representative data. The FPGA-based feature extraction implementation allows the system to process data from 30 acoustic sensors in real time with an affordable cost. Finally, several open questions derived from this research are discussed to be considered for future works.

Automated Sound Recognition Provides Insights into the Behavioral Ecology of a Tropical Bird

Computer-assisted species recognition facilitates the analysis of relevant biological information in continuous audio recordings. In the present study, we assess the suitability of this approach for determining distinct life-cycle phases of the Southern Lapwing Vanellus chilensis lampronotus based on adult vocal activity. For this purpose we use passive 14-min and 30-min soundscape recordings (n = 33 201) collected in 24/7 mode between November 2012 and October 2013 in Brazil’s Pantanal wetlands. Time-stamped detections of V. chilensis call events (n = 62 292) were obtained with a species-specific sound recognizer. We demonstrate that the breeding season fell in a three-month period from mid-May to early August 2013, between the end of the flood cycle and the height of the dry season. Several phases of the lapwing’s life history were identified with presumed error margins of a few days: pre-breeding, territory establishment and egg-laying, incubation, hatching, parental defense of chicks, and post-breeding. Diurnal time budgets confirm high acoustic activity levels during midday hours in June and July, indicative of adults defending young. By August, activity patterns had reverted to nonbreeding mode, with peaks around dawn and dusk and low call frequency during midday heat. We assess the current technological limitations of the V. chilensis recognizer through a comprehensive performance assessment and scrutinize the usefulness of automated acoustic recognizers in studies on the distribution pattern, ecology, life history, and conservation status of sound-producing animal species.

Fully-automated identification of fish species based on otolith contour: using short-time Fourier transform and discriminant analysis (STFT-DA)

Background. Fish species may be identified based on their unique otolith shape or contour. Several pattern recognition methods have been proposed to classify fish species through morphological features of the otolith contours. However, there has been no fully-automated species identification model with the accuracy higher than 80%. The purpose of the current study is to develop a fully-automated model, based on the otolith contours, to identify the fish species with the high classification accuracy.

Methods. Images of the right sagittal otoliths of 14 fish species from three families namely Sciaenidae, Ariidae, and Engraulidae were used to develop the proposed identification model. Short-time Fourier transform (STFT) was used, for the first time in the area of otolith shape analysis, to extract important features of the otolith contours. Discriminant Analysis (DA), as a classification technique, was used to train and test the model based on the extracted features.

Results. Performance of the model was demonstrated using species from three families separately, as well as all species combined. Overall classification accuracy of the model was greater than 90% for all cases. In addition, effects of STFT variables on the performance of the identification model were explored in this study.

Conclusions. Short-time Fourier transform could determine important features of the otolith outlines. The fully-automated model proposed in this study (STFT-DA) could predict species of an unknown specimen with acceptable identification accuracy. The model codes can be accessed at http://mybiodiversityontologies.um.edu.my/Otolith/ and https://peerj.com/preprints/1517/. The current model has flexibility to be used for more species and families in future studies.

Automatic large-scale classification of bird sounds is strongly improved by unsupervised feature learning

Automatic species classification of birds from their sound is a computational tool of increasing importance in ecology, conservation monitoring and vocal communication studies. To make classification useful in practice, it is crucial to improve its accuracy while ensuring that it can run at big data scales. Many approaches use acoustic measures based on spectrogram-type data, such as the Mel-frequency cepstral coefficient (MFCC) features which represent a manually-designed summary of spectral information. However, recent work in machine learning has demonstrated that features learnt automatically from data can often outperform manually-designed feature transforms. Feature learning can be performed at large scale and “unsupervised”, meaning it requires no manual data labelling, yet it can improve performance on “supervised” tasks such as classification. In this work we introduce a technique for feature learning from large volumes of bird sound recordings, inspired by techniques that have proven useful in other domains. We experimentally compare twelve different feature representations derived from the Mel spectrum (of which six use this technique), using four large and diverse databases of bird vocalisations, classified using a random forest classifier. We demonstrate that in our classification tasks, MFCCs can often lead to worse performance than the raw Mel spectral data from which they are derived. Conversely, we demonstrate that unsupervised feature learning provides a substantial boost over MFCCs and Mel spectra without adding computational complexity after the model has been trained. The boost is particularly notable for single-label classification tasks at large scale. The spectro-temporal activations learned through our procedure resemble spectro-temporal receptive fields calculated from avian primary auditory forebrain. However, for one of our datasets, which contains substantial audio data but few annotations, increased performance is not discernible. We study the interaction between dataset characteristics and choice of feature representation through further empirical analysis.