Machine learning enables improved runtime and precision for bio-loggers on seabirds

Time synchronisation for millisecond-precision on bio-loggers

Time-synchronised data streams from bio-loggers are becoming increasingly important for analysing and interpreting intricate animal behaviour including split-second decision making, group dynamics, and collective responses to environmental conditions. With the increased use of AI-based approaches for behaviour classification, time synchronisation between recording systems is becoming an essential challenge. Current solutions in bio-logging rely on manually removing time errors during post processing, which is complex and typically does not achieve sub-second timing accuracies.We first introduce an error model to quantify time errors, then optimise three wireless methods for automated onboard time (re)synchronisation on bio-loggers (GPS, WiFi, proximity messages). The methods can be combined as required and, when coupled with a state-of-the-art real time clock, facilitate accurate time annotations for all types of bio-logging data without need for post processing. We analyse time accuracy of our optimised methods in stationary tests and in a case study on 99 Egyptian fruit bats (Rousettus aegyptiacus). Based on the results, we offer recommendations for projects that require high time synchrony.During stationary tests, our low power synchronisation methods achieved median time accuracies of 2.72 / 0.43 ms (GPS / WiFi), compared to UTC time, and relative median time accuracies of 5 ms between tags (wireless proximity messages). In our case study with bats, we achieved a median relative time accuracy of 40 ms between tags throughout the entire 10-day duration of tag deployment. Using only one automated resynchronisation per day, permanent UTC time accuracies of ≤ 185 ms can be guaranteed in 95% of cases over a wide temperature range between 0 and 50 °C. Accurate timekeeping required a minimal battery capacity, operating in the nano- to microwatt range.Time measurements on bio-loggers, similar to other forms of sensor-derived data, are prone to errors and so far received little scientific attention. Our combinable methods offer a means to quantify time errors and autonomously correct them at the source (i.e., on bio-loggers). This approach facilitates sub-second comparisons of simultaneously recorded time series data across multiple individuals and off-animal devices such as cameras or weather stations. Through automated resynchronisations on bio-loggers, long-term sub-second accurate timestamps become feasible, even for life-time studies on animals. We contend that our methods have potential to greatly enhance the quality of ecological data, thereby improving scientific conclusions.

Hidden rivals: the negative impacts of dolphinfish on seabird foraging behaviour

Marine predators often aggregate at the air-sea boundary layer to pursue shared prey. In such scenarios, seabirds are likely to benefit from underwater predators herding fish schools into tight clusters thereby enhancing seabirds' prey detectability and capture potential. However, this coexistence can lead to competition, affecting not only immediate foraging strategies but also their distribution and interspecies dynamics. We investigated both the longitudinal relationships and instantaneous interactions between streaked shearwaters (Calonectris leucomelas) and common dolphinfish (Coryphaena hippurus), both preying on Japanese anchovy (Engraulis japonicus). Using GPS data from 2011 to 2021, we calculated behavioural parameters for streaked shearwaters as an index of time spent and distance travelled. Despite the abundance of Japanese anchovies, we found that streaked shearwaters might increase their foraging time in the presence of underwater predators. Moreover, video loggers provided direct evidence of streaked shearwaters and common dolphinfish attacking the same fish schools, potentially interfering with bird foraging by dolphinfish. Our results suggest that the presence of underwater predators in a given patch might increase the time spent by seabirds foraging without affecting the distance travelled. This highlights the need for future studies that consider the potential adverse effects of other top predators on seabird prey availability.

Edge computing in wildlife behavior and ecology

Modern sensor technologies increasingly enrich studies in wildlife behavior and ecology. However, constraints on weight, connectivity, energy and memory availability limit their implementation. With the advent of edge computing, there is increasing potential to mitigate these constraints, and drive major advancements in wildlife studies.

Automatic recording of rare behaviors of wild animals using video bio-loggers with on-board light-weight outlier detector

Rare behaviors displayed by wild animals can generate new hypotheses; however, observing such behaviors may be challenging. While recent technological advancements, such as bio-loggers, may assist in documenting rare behaviors, the limited running time of battery-powered bio-loggers is insufficient to record rare behaviors when employing high-cost sensors (e.g. video cameras). In this study, we propose an artificial intelligence (AI)-enabled bio-logger that automatically detects outlier readings from always-on low-cost sensors, e.g. accelerometers, indicative of rare behaviors in target animals, without supervision by researchers, subsequently activating high-cost sensors to record only these behaviors. We implemented an on-board outlier detector via knowledge distillation by building a lightweight outlier classifier supervised by a high-cost outlier behavior detector trained in an unsupervised manner. The efficacy of AI bio-loggers has been demonstrated on seabirds, where videos and sensor data captured by the bio-loggers have enabled the identification of some rare behaviors, facilitating analyses of their frequency, and potential factors underlying these behaviors. This approach offers a means of documenting previously overlooked rare behaviors, augmenting our understanding of animal behavior.

The role of individual variability on the predictive performance of machine learning applied to large bio-logging datasets

Animal-borne tagging (bio-logging) generates large and complex datasets. In particular, accelerometer tags, which provide information on behaviour and energy expenditure of wild animals, produce high-resolution multi-dimensional data, and can be challenging to analyse. We tested the performance of commonly used artificial intelligence tools on datasets of increasing volume and dimensionality. By collecting bio-logging data across several sampling seasons, datasets are inherently characterized by inter-individual variability. Such information should be considered when predicting behaviour. We integrated both unsupervised and supervised machine learning approaches to predict behaviours in two penguin species. The classified behaviours obtained from the unsupervised approach Expectation Maximisation were used to train the supervised approach Random Forest. We assessed agreement between the approaches, the performance of Random Forest on unknown data and the implications for the calculation of energy expenditure. Consideration of behavioural variability resulted in high agreement (> 80%) in behavioural classifications and minimal differences in energy expenditure estimates. However, some outliers with < 70% of agreement, highlighted how behaviours characterized by signal similarity are confused. We advise the broad bio-logging community, approaching these large datasets, to be cautious when upscaling predictions, as this might lead to less accurate estimates of behaviour and energy expenditure.

A Review of Neurologgers for Extracellular Recording of Neuronal Activity in the Brain of Freely Behaving Wild Animals

Simultaneous monitoring of animal behavior and neuronal activity in the brain enables us to examine the neural underpinnings of behaviors. Conventionally, the neural activity data are buffered, amplified, multiplexed, and then converted from analog to digital in the head-stage amplifier, following which they are transferred to a storage server via a cable. Such tethered recording systems, intended for indoor use, hamper the free movement of animals in three-dimensional (3D) space as well as in large spaces or underwater, making it difficult to target wild animals active under natural conditions; it also presents challenges in realizing its applications to humans, such as the Brain-Machine Interfaces (BMI). Recent advances in micromachine technology have established a wireless logging device called a neurologger, which directly stores neural activity on ultra-compact memory media. The advent of the neurologger has triggered the examination of the neural correlates of 3D flight, underwater swimming of wild animals, and translocation experiments in the wild. Examples of the use of neurologgers will provide an insight into understanding the neural underpinnings of behaviors in the natural environment and contribute to the practical application of BMI. Here we outline the monitoring of the neural underpinnings of flying and swimming behaviors using neurologgers. We then focus on neuroethological findings and end by discussing their future perspectives.

Machine learning approaches for biomolecular, biophysical, and biomaterials research

A fluent conversation with a virtual assistant, person-tailored news feeds, and deep-fake images created within seconds-all those things that have been unthinkable for a long time are now a part of our everyday lives. What these examples have in common is that they are realized by different means of machine learning (ML), a technology that has fundamentally changed many aspects of the modern world. The possibility to process enormous amount of data in multi-hierarchical, digital constructs has paved the way not only for creating intelligent systems but also for obtaining surprising new insight into many scientific problems. However, in the different areas of biosciences, which typically rely heavily on the collection of time-consuming experimental data, applying ML methods is a bit more challenging: Here, difficulties can arise from small datasets and the inherent, broad variability, and complexity associated with studying biological objects and phenomena. In this Review, we give an overview of commonly used ML algorithms (which are often referred to as "machines") and learning strategies as well as their applications in different bio-disciplines such as molecular biology, drug development, biophysics, and biomaterials science. We highlight how selected research questions from those fields were successfully translated into machine readable formats, discuss typical problems that can arise in this context, and provide an overview of how to resolve those encountered difficulties.

Estimation of the Maternal Investment of Sea Turtles by Automatic Identification of Nesting Behavior and Number of Eggs Laid from a Tri-Axial Accelerometer

Simple Summary

During the reproduction period, female sea turtles come several times onto the beaches to lay their eggs. Monitoring of the nesting populations is therefore important to estimate the state of a population and its future. However, measuring the clutch size and frequency of sea turtles is tedious work that requires rigorous monitoring of the nesting site throughout the breeding season. In order to support the fieldwork, we propose an automatic method to remotely record the behavior on land of the sea turtles from animal-attached sensors; an accelerometer. The proposed method estimates, with an accuracy of 95%, the behaviors on land of sea turtles and the number of eggs laid. This automatic method should therefore help researchers monitor nesting sea turtle populations and contribute to improving global knowledge on the demographic status of these threatened species.

Abstract

Monitoring reproductive outputs of sea turtles is difficult, as it requires a large number of observers patrolling extended beaches every night throughout the breeding season with the risk of missing nesting individuals. We introduce the first automatic method to remotely record the reproductive outputs of green turtles (Chelonia mydas) using accelerometers. First, we trained a fully convolutional neural network, the V-net, to automatically identify the six behaviors shown during nesting. With an accuracy of 0.95, the V-net succeeded in detecting the Egg laying process with a precision of 0.97. Then, we estimated the number of laid eggs from the predicted Egg laying sequence and obtained the outputs with a mean relative error of 7% compared to the observed numbers in the field. Based on deployment of non-invasive and miniature loggers, the proposed method should help researchers monitor nesting sea turtle populations. Furthermore, its use can be coupled with the deployment of accelerometers at sea during the intra-nesting period, from which behaviors can also be estimated. The knowledge of the behavior of sea turtle on land and at sea during the entire reproduction period is essential to improve our knowledge of this threatened species.

Perspectives in machine learning for wildlife conservation

Inexpensive and accessible sensors are accelerating data acquisition in animal ecology. These technologies hold great potential for large-scale ecological understanding, but are limited by current processing approaches which inefficiently distill data into relevant information. We argue that animal ecologists can capitalize on large datasets generated by modern sensors by combining machine learning approaches with domain knowledge. Incorporating machine learning into ecological workflows could improve inputs for ecological models and lead to integrated hybrid modeling tools. This approach will require close interdisciplinary collaboration to ensure the quality of novel approaches and train a new generation of data scientists in ecology and conservation.

Quantifying song behavior in a free-living, light-weight, mobile bird using accelerometers

To acquire a fundamental understanding of animal communication, continuous observations in a natural setting and at an individual level are required. Whereas the use of animal-borne acoustic recorders in vocal studies remains challenging, light-weight accelerometers can potentially register individuals' vocal output when this coincides with body vibrations. We collected one-dimensional accelerometer data using light-weight tags on a free-living, crepuscular bird species, the European Nightjar (Caprimulgus europaeus). We developed a classification model to identify four behaviors (rest, sing, fly, and leap) from accelerometer data and, for the purpose of this study, validated the classification of song behavior. Male nightjars produce a distinctive "churring" song while they rest on a stationary song post. We expected churring to be associated with body vibrations (i.e., medium-amplitude body acceleration), which we assumed would be easy to distinguish from resting (i.e., low-amplitude body acceleration). We validated the classification of song behavior using simultaneous GPS tracking data (i.e., information on individuals' movement and proximity to audio recorders) and vocal recordings from stationary audio recorders at known song posts of one tracked individual. Song activity was detected by the classification model with an accuracy of 92%. Beyond a threshold of 20 m from the audio recorders, only 8% of the classified song bouts were recorded. The duration of the detected song activity (i.e., acceleration data) was highly correlated with the duration of the simultaneously recorded song bouts (correlation coefficient = 0.87, N = 10, S = 21.7, p = .001). We show that accelerometer-based identification of vocalizations could serve as a promising tool to study communication in free-living, small-sized birds and demonstrate possible limitations of audio recorders to investigate individual-based variation in song behavior.

R package for animal behavior classification from accelerometer data-rabc

Increasingly, animal behavior studies are enhanced through the use of accelerometry. To allow translation of raw accelerometer data to animal behaviors requires the development of classifiers. Here, we present the "rabc" (r for animal behavior classification) package to assist researchers with the interactive development of such animal behavior classifiers in a supervised classification approach. The package uses datasets consisting of accelerometer data with their corresponding animal behaviors (e.g., for triaxial accelerometer data along the x, y and z axes arranged as "x, y, z, x, y, z,…, behavior"). Using an example dataset collected on white stork (Ciconia ciconia), we illustrate the workflow of this package, including accelerometer data visualization, feature calculation, feature selection, feature visualization, extreme gradient boost model training, validation, and, finally, a demonstration of the behavior classification results.

Video and acceleration records of streaked shearwaters allows detection of two foraging behaviours associated with large marine predators

The study of seabird behaviour has largely relied on animal-borne tags to gather information, requiring interpretation to estimate at-sea behaviours. Details of shallow-diving birds' foraging are less known than deep-diving species due to difficulty in identifying shallow dives from biologging devices. Development of smaller video loggers allow a direct view of these birds' behaviours, at the cost of short battery capacity. However, recordings from video loggers combined with relatively low power usage accelerometers give a means to develop a reliable foraging detection method. Combined video and acceleration loggers were attached to streaked shearwaters in Funakoshi-Ohshima Island (39°24'N,141°59'E) during the breeding season in 2018. Video recordings were classified into behavioural categories (rest, transit, and foraging) and a detection method was generated from the acceleration signals. Two foraging behaviours, surface seizing and foraging dives, are reported with video recordings. Surface seizing was comprised of successive take-offs and landings (mean duration 0.6 and 1.5s, respectively), while foraging dives were shallow subsurface dives (3.2s mean duration) from the air and water surface. Birds were observed foraging close to marine predators, including dolphins and large fish. Results of the behaviour detection method were validated against video recordings, with mean true and false positive rates of 90% and 0%, 79% and 5%, and 66% and <1%, for flight, surface seizing, and foraging dives, respectively. The detection method was applied to longer duration acceleration and GPS datasets collected during the 2018 and 2019 breeding seasons. Foraging trips lasted between 1 - 8 days, with birds performing on average 16 surface seizing events and 43 foraging dives per day, comprising <1% of daily activity, while transit and rest took up 55 and 40%, respectively. This foraging detection method can address the difficulties of recording shallow-diving foraging behaviour and provides a means to measure activity budgets across shallow diving seabird species.

Seabird Biologging System with Compact Waterproof Airflow Sensor

This paper presents a seabird biologging system with a compact waterproof airflow sensor. Although biologging methods have attracted attention in the evaluation of seabird flight performance, a direct measurement method of airflow velocity has not yet been established. When an airflow sensor is added to a biologging system, a more accurate assessment of the flight performance can be obtained. We developed a compact Pitot tube-type airflow sensor that is specialized for seabird biologging systems. Here, we integrated micro electro mechanical system (MEMS) sensor chips and a sensing circuit into the Pitot tube housing. Then, we conducted a wind tunnel experiment using a stuffed seabird and the fabricated sensor. The results confirmed that the sensor responds to the wind speed even when attached to the dorsal surface of the seabird. Based on the above, we believe that the proposed sensor can be applied to practical seabird biologging systems.

Development of Data Logger Separator for Bio-Logging of Wild Seabirds

The bio-logging technique is extensively used in the fields of ecology and ethology, wherein a data logger, such as a sensor or camera, is attached to the target animal’s body to collect the required data. In this method, the efficiency of recovery of the data logger is not ideal. In this study, we proposed a new recovery method, with the aim of addressing the aforementioned problem in bio-logging. The authors previously fabricated a data-logger separator, which weighed approximately 10 g, and was targeted at small seabirds. Because there were some problems associated with the circuit board and the separation performance of this device, we modified the device to overcome the previous drawbacks. We fabricated a flexible printed circuit to improve the operation of the mounted actuator and wireless microcomputer, and improve the efficiency of the fabrication process. We conducted an experiment to determine the proper length and position at which the actuator is attached, in order to achieve a stable motion. We thus fabricated a new prototype with these improvements and performed an operational test at low temperatures from a particular distance, simulating actual usage in a natural environment. The results demonstrated that separation occurred without failure, thus indicating that the separator can be efficiently used in practical environment.

An evaluation of machine learning classifiers for next-generation, continuous-ethogram smart trackers

BACKGROUND

Our understanding of movement patterns and behaviours of wildlife has advanced greatly through the use of improved tracking technologies, including application of accelerometry (ACC) across a wide range of taxa. However, most ACC studies either use intermittent sampling that hinders continuity or continuous data logging relying on tracker retrieval for data downloading which is not applicable for long term study. To allow long-term, fine-scale behavioural research, we evaluated a range of machine learning methods for their suitability for continuous on-board classification of ACC data into behaviour categories prior to data transmission.

METHODS

We tested six supervised machine learning methods, including linear discriminant analysis (LDA), decision tree (DT), support vector machine (SVM), artificial neural network (ANN), random forest (RF) and extreme gradient boosting (XGBoost) to classify behaviour using ACC data from three bird species (white stork Ciconia ciconia, griffon vulture Gyps fulvus and common crane Grus grus) and two mammals (dairy cow Bos taurus and roe deer Capreolus capreolus).

RESULTS

Using a range of quality criteria, SVM, ANN, RF and XGBoost performed well in determining behaviour from ACC data and their good performance appeared little affected when greatly reducing the number of input features for model training. On-board runtime and storage-requirement tests showed that notably ANN, RF and XGBoost would make suitable on-board classifiers.

CONCLUSIONS

Our identification of using feature reduction in combination with ANN, RF and XGBoost as suitable methods for on-board behavioural classification of continuous ACC data has considerable potential to benefit movement ecology and behavioural research, wildlife conservation and livestock husbandry.

Remote Recognition of Moving Behaviors for Captive Harbor Seals Using a Smart-Patch System via Bluetooth Communication

Animal telemetry has been recognized as a core platform for exploring animal species due to future opportunities in terms of its contribution toward marine fisheries and living resources. Herein, biologging systems with pressure sensors are successfully implemented via open-source hardware platforms, followed by immediate application to captive harbor seals (HS). Remotely captured output voltage signals in real-time mode via Bluetooth communication were reproducibly and reliably recorded on the basis of hours using a smartphone built with data capturing software with graphic user interface (GUI). Output voltages, corresponding to typical behaviors on the captive HS, such as stopping (A), rolling (B), flapping (C), and sliding (D), are clearly obtained, and their analytical interpretation on captured electrical signals are fully validated via a comparison study with consecutively captured images for each motion of the HS. Thus, the biologging system with low cost and light weight, which is fully compatible with a conventional smartphone, is expected to potentially contribute toward future anthology of seal animals.