An Automated Pipeline for Image Processing and Data Treatment to Track Activity Rhythms of Paragorgia arborea in Relation to Hydrographic Conditions

New interactive machine learning tool for marine image analysis

Advancing imaging technologies are drastically increasing the rate of marine video and image data collection. Often these datasets are not analysed to their full potential as extracting information for multiple species is incredibly time-consuming. This study demonstrates the capability of the open-source interactive machine learning tool, RootPainter, to analyse large marine image datasets quickly and accurately. The ability of RootPainter to extract the presence and surface area of the cold-water coral reef associate sponge species, Mycale lingua, was tested in two datasets: 18 346 time-lapse images and 1420 remotely operated vehicle video frames. New corrective annotation metrics integrated with RootPainter allow objective assessment of when to stop model training and reduce the need for manual model validation. Three highly accurate M. lingua models were created using RootPainter, with an average dice score of 0.94 ± 0.06. Transfer learning aided the production of two of the models, increasing analysis efficiency from 6 to 16 times faster than manual annotation for time-lapse images. Surface area measurements were extracted from both datasets allowing future investigation of sponge behaviours and distributions. Moving forward, interactive machine learning tools and model sharing could dramatically increase image analysis speeds, collaborative research and our understanding of spatiotemporal patterns in biodiversity.

A data science approach for multi-sensor marine observatory data monitoring cold water corals (Paragorgia arborea) in two campaigns

Fixed underwater observatories (FUO), equipped with digital cameras and other sensors, become more commonly used to record different kinds of time series data for marine habitat monitoring. With increasing numbers of campaigns, numbers of sensors and campaign time, the volume and heterogeneity of the data, ranging from simple temperature time series to series of HD images or video call for new data science approaches to analyze the data. While some works have been published on the analysis of data from one campaign, we address the problem of analyzing time series data from two consecutive monitoring campaigns (starting late 2017 and late 2018) in the same habitat. While the data from campaigns in two separate years provide an interesting basis for marine biology research, it also presents new data science challenges, like the the marine image analysis in data form more than one campaign. In this paper, we analyze the polyp activity of two Paragorgia arborea cold water coral (CWC) colonies using FUO data collected from November 2017 to June 2018 and from December 2018 to April 2019. We successfully apply convolutional neural networks (CNN) for the segmentation and classification of the coral and the polyp activities. The result polyp activity data alone showed interesting temporal patterns with differences and similarities between the two time periods. A one month "sleeping" period in spring with almost no activity was observed in both coral colonies, but with a shift of approximately one month. A time series prediction experiment allowed us to predict the polyp activity from the non-image sensor data using recurrent neural networks (RNN). The results pave a way to a new multi-sensor monitoring strategy for Paragorgia arborea behaviour.

Image dataset for benchmarking automated fish detection and classification algorithms

Multiparametric video-cabled marine observatories are becoming strategic to monitor remotely and in real-time the marine ecosystem. Those platforms can achieve continuous, high-frequency and long-lasting image data sets that require automation in order to extract biological time series. The OBSEA, located at 4 km from Vilanova i la Geltrú at 20 m depth, was used to produce coastal fish time series continuously over the 24-h during 2013-2014. The image content of the photos was extracted via tagging, resulting in 69917 fish tags of 30 taxa identified. We also provided a meteorological and oceanographic dataset filtered by a quality control procedure to define real-world conditions affecting image quality. The tagged fish dataset can be of great importance to develop Artificial Intelligence routines for the automated identification and classification of fishes in extensive time-lapse image sets.

Phenology in the deep sea: seasonal and tidal feeding rhythms in a keystone octocoral

Biological rhythms are widely known in terrestrial and marine systems, where the behaviour or function of organisms may be tuned to environmental variation over periods from minutes to seasons or longer. Although well characterized in coastal environments, phenology remains poorly understood in the deep sea. Here we characterized intra-annual dynamics of feeding activity for the deep-sea octocoral Paragorgia arborea. Hourly changes in polyp activity were quantified using a time-lapse camera deployed for a year on Sur Ridge (1230 m depth; Northeast Pacific). The relationship between feeding and environmental variables, including surface primary production, temperature, acoustic backscatter, current speed and direction, was evaluated. Feeding activity was highly seasonal, with a dormancy period identified between January and early April, reflecting seasonal changes in food availability as suggested by primary production and acoustic backscatter data. Moreover, feeding varied with tides, which likely affected food delivery through cyclic oscillation in current speed and direction. This study provides the first evidence of behavioural rhythms in a coral species at depth greater than 1 km. Information on the feeding biology of this cosmopolitan deep-sea octocoral will contribute to a better understanding of how future environmental change may affect deep-sea coral communities and the ecosystem services they provide.

A low-cost, long-term underwater camera trap network coupled with deep residual learning image analysis

Understanding long-term trends in marine ecosystems requires accurate and repeatable counts of fishes and other aquatic organisms on spatial and temporal scales that are difficult or impossible to achieve with diver-based surveys. Long-term, spatially distributed cameras, like those used in terrestrial camera trapping, have not been successfully applied in marine systems due to limitations of the aquatic environment. Here, we develop methodology for a system of low-cost, long-term camera traps (Dispersed Environment Aquatic Cameras), deployable over large spatial scales in remote marine environments. We use machine learning to classify the large volume of images collected by the cameras. We present a case study of these combined techniques’ use by addressing fish movement and feeding behavior related to halos, a well-documented benthic pattern in shallow tropical reefscapes. Cameras proved able to function continuously underwater at deployed depths (up to 7 m, with later versions deployed to 40 m) with no maintenance or monitoring for over five months and collected a total of over 100,000 images in time-lapse mode (by 15 minutes) during daylight hours. Our ResNet-50-based deep learning model achieved 92.5% overall accuracy in sorting images with and without fishes, and diver surveys revealed that the camera images accurately represented local fish communities. The cameras and machine learning classification represent the first successful method for broad-scale underwater camera trap deployment, and our case study demonstrates the cameras’ potential for addressing questions of marine animal behavior, distributions, and large-scale spatial patterns.

Autonomous Surface and Underwater Vehicles as Effective Ecosystem Monitoring and Research Platforms in the Arctic-The Glider Project

Effective ocean management requires integrated and sustainable ocean observing systems enabling us to map and understand ecosystem properties and the effects of human activities. Autonomous subsurface and surface vehicles, here collectively referred to as “gliders”, are part of such ocean observing systems providing high spatiotemporal resolution. In this paper, we present some of the results achieved through the project “Unmanned ocean vehicles, a flexible and cost-efficient offshore monitoring and data management approach—GLIDER”. In this project, three autonomous surface and underwater vehicles were deployed along the Lofoten–Vesterålen (LoVe) shelf-slope-oceanic system, in Arctic Norway. The aim of this effort was to test whether gliders equipped with novel sensors could effectively perform ecosystem surveys by recording physical, biogeochemical, and biological data simultaneously. From March to September 2018, a period of high biological activity in the area, the gliders were able to record a set of environmental parameters, including temperature, salinity, and oxygen, map the spatiotemporal distribution of zooplankton, and record cetacean vocalizations and anthropogenic noise. A subset of these parameters was effectively employed in near-real-time data assimilative ocean circulation models, improving their local predictive skills. The results presented here demonstrate that autonomous gliders can be effective long-term, remote, noninvasive ecosystem monitoring and research platforms capable of operating in high-latitude marine ecosystems. Accordingly, these platforms can record high-quality baseline environmental data in areas where extractive activities are planned and provide much-needed information for operational and management purposes.