ERISNet: deep neural network for Sargassum detection along the coastline of the Mexican Caribbean

Towards sustainable coastal management: aerial imagery and deep learning for high-resolution Sargassum mapping

The massive arrival of pelagic Sargassum on the coasts of several countries of the Atlantic Ocean began in 2011 and to date continues to generate social and environmental challenges for the region. Therefore, knowing the distribution and quantity of Sargassum in the ocean, coasts, and beaches is necessary to understand the phenomenon and develop protocols for its management, use, and final disposal. In this context, the present study proposes a methodology to calculate the area Sargassum occupies on beaches in square meters, based on the semantic segmentation of aerial images using the pix2pix architecture. For training and testing the algorithm, a unique dataset was built from scratch, consisting of 15,268 aerial images segmented into three classes. The images correspond to beaches in the cities of Mahahual and Puerto Morelos, located in Quintana Roo, Mexico. To analyze the results the fβ-score metric was used. The results for the Sargassum class indicate that there is a balance between false positives and false negatives, with a slight bias towards false negatives, which means that the algorithm tends to underestimate the Sargassum pixels in the images. To know the confidence intervals within which the algorithm performs better, the results of the f0.5-score metric were resampled by bootstrapping considering all classes and considering only the Sargassum class. From the above, we found that the algorithm offers better performance when segmenting Sargassum images on the sand. From the results, maps showing the Sargassum coverage area along the beach were designed to complement the previous ones and provide insight into the field of study.

Innovative spectral characterisation of beached pelagic sargassum towards remote estimation of biochemical and phenotypic properties

In recent years, pelagic sargassum (S. fluitans and S. natans - henceforth sargassum) macroalgal blooms have become more frequent and larger with higher biomass in the Tropical Atlantic region. They have environmental and socio-economic impacts, particularly on coastal ecosystems, tourism, fisheries and aquaculture industries, and on public health. Despite these challenges, sargassum biomass has the potential to offer commercial opportunities in the blue economy, although, it is reliant on key chemical and physical characteristics of the sargassum for specific use. In this study, we aim to utilise remotely sensed spectral profiles to determine species/morphotypes at different decomposition stages and their biochemical composition to support monitoring and valorisation of sargassum. For this, we undertook dedicated field campaigns in Barbados and Ghana to collect, for the first time, in situ spectral measurements between 350 and 2500 nm using a Spectra Vista Corp (SVC) HR-1024i field spectrometer of pelagic sargassum stranded biomass. The spectral measurements were complemented by uncrewed aerial system surveys using a DJI Phantom 4 drone and a DJI P4 multispectral instrument. Using the ground and airborne datasets this research developed an operational framework for remote detection of beached sargassum; and created spectral profiles of species/morphotypes and decomposition maps to infer biochemical composition. We were able to identify some key spectral regions, including a consistent absorption feature (920-1080 nm) found in all of the sargassum morphotype spectral profiles; we also observed distinction between fresh and recently beached sargassum particularly around 900-1000 nm. This work can support pelagic sargassum management and contribute to effective utilisation of the sargassum biomass to ultimately alleviate some of the socio-economic impacts associated with this emerging environmental challenge.

SLWE-Net: An improved lightweight U-Net for Sargassum extraction from GOCI images

The Sargassum bloom has severely impacted the ecological environment of the East China Sea and the Yellow Sea, causing significant economic losses. In recent years, deep learning has seen extensive development due to its outstanding feature extraction capabilities. However, the deep learning process typically involves a large number of parameters and computations. To address this issue, this paper proposes a lightweight deep learning network based on the U-Net framework, called SLWE-NET, which uses lightweight modules to replace the feature extraction modules in U-Net. In this experiment, SLWE-Net performed the best in both extraction accuracy and model lightweight. Compared to the formal U-Net, the number of parameters decreased by 65.83 %, the model size reduced from 94.97 MB to 32.51 MB, and the mIoU increased to 93.81 %. Therefore, the method proposed in this paper is beneficial for Sargassum extraction and provides a basis for operational monitoring.

Long-term spatial variation of algal blooms extracted using the U-net model from 10 years of GOCI imagery in the East China Sea

Long-term satellite missions could help to provide insights into spatial and temporal variations in algal blooms. However, the traditional reflectance-based method has limitations in regards to determining the available threshold for algal bloom detection among the time-varying observation conditions. In terms of extracting useful information from long-term data series precisely and efficiently, the deep learning method has shown its superiority over traditional algorithms in batch data processing. In this study, a U-net model for algal bloom extraction along the coast of the East China Sea was developed using GOCI images. The U-net model was trained with two different datasets that were constructed with six-band channels (all visible bands from GOCI imagery) and RGB-band channels (bands of 443, 555, and 680 nm from GOCI imagery). The quantitative assessment from the U-net models suggests that the U-net model trained with the six-band channel datasets outperformed the RGB-band channel datasets, with increases of 23.6%, 18.1%, and 12.5% in terms of accuracy, precision, and F-score, respectively. The validation map derived from the U-net model trained with six-band channel datasets also showed considerable matching with the ground-truth maps. By using the U-net model, the occurrence of algal blooms was automatically extracted from GOCI images. A 10-year time series of GOCI data collected between 2011 and 2020 was derived using an output-trained U-net model to explore spatial variation along the coast of the ECS. It was found that the most affected areas of the algal blooms varied by year, but were mainly located in the Zhoushan and Zhejiang coasts. Additionally, by performing principal component analysis on the daily meteorological data during April and August 2011-2020, factors related to algal bloom occurrence were discussed.

Use of semantic segmentation for mapping Sargassum on beaches

The unusual arrival of Sargassum on Caribbean beaches is an emerging problem that has generated numerous challenges. The monitoring, visualization, and estimation of Sargassum coverage on the beaches remain a constant complication. This study proposes a new mapping methodology to estimate Sargassum coverage on the beaches. Semantic segmentation of geotagged photographs allows the generation of accurate maps showing the percent coverage of Sargassum. The first dataset of segmented Sargassum images was built for this study and used to train the proposed model. The results demonstrate that the currently proposed method has an accuracy of 91%, improving on the results reported in the state-of-the-art method where data was also collected through a crowdsourcing scheme, in which only information on the presence and absence of Sargassum is displayed.

SRSe-Net: Super-Resolution-Based Semantic Segmentation Network for Green Tide Extraction

Due to the phenomenon of mixed pixels in low-resolution remote sensing images, the green tide spectral features with low Enteromorpha coverage are not obvious. Super-resolution technology based on deep learning can supplement more detailed information for subsequent semantic segmentation tasks. In this paper, a novel green tide extraction method for MODIS images based on super-resolution and a deep semantic segmentation network was proposed. Inspired by the idea of transfer learning, a super-resolution model (i.e., WDSR) is first pre-trained with high spatial resolution GF1-WFV images, and then the representations learned in the GF1-WFV image domain are transferred to the MODIS image domain. The improvement of remote sensing image resolution enables us to better distinguish the green tide patches from the surrounding seawater. As a result, a deep semantic segmentation network (SRSe-Net) suitable for large-scale green tide information extraction is proposed. The SRSe-Net introduced the dense connection mechanism on the basis of U-Net and replaces the convolution operations with dense blocks, which effectively obtained the detailed green tide boundary information by strengthening the propagation and reusing features. In addition, the SRSe-Net reducs the pooling layer and adds a bridge module in the final stage of the encoder. The experimental results show that a SRSe-Net can obtain more accurate segmentation results with fewer network parameters.

A Deep Learning and GIS Approach for the Optimal Positioning of Wave Energy Converters

Renewable Energy Sources provide a viable solution to the problem of ever-increasing climate change. For this reason, several countries focus on electricity production using alternative sources. In this paper, the optimal positioning of the installation of wave energy converters is examined taking into account geospatial and technical limitations. Geospatial constraints depend on Land Use classes and seagrass of the coastal areas, while technical limitations include meteorological conditions and the morphology of the seabed. Suitable installation areas are selected after the exclusion of points that do not meet the aforementioned restrictions. We implemented a Deep Neural Network that operates based on heterogeneous data fusion, in this case satellite images and time series of meteorological data. This fact implies the definition of a two-branches architecture. The branch that is trained with image data provides for the localization of dynamic geospatial classes in the potential installation area, whereas the second one is responsible for the classification of the region according to the potential wave energy using wave height and period time series. In making the final decision on the suitability of the potential area, a large number of static land use data play an important role. These data are combined with neural network predictions for the optimizing positioning of the Wave Energy Converters. For the sake of completeness and flexibility, a Multi-Task Neural Network is developed. This model, in addition to predicting the suitability of an area depending on seagrass patterns and wave energy, also predicts land use classes through Multi-Label classification process. The proposed methodology is applied in the marine area of the city of Sines, Portugal. The first neural network achieves 98.7% Binary Classification accuracy, while the Multi-Task Neural Network 97.5% in the same metric and 93.5% in the F1 score of the Multi-Label classification output.

Collective view: mapping Sargassum distribution along beaches

The atypical arrival of pelagic Sargassum to the Mexican Caribbean beaches has caused considerable economic and ecological damage. Furthermore, it has raised new challenges for monitoring the coastlines. Historically, satellite remote-sensing has been used for Sargassum monitoring in the ocean; nonetheless, limitations in the temporal and spatial resolution of available satellite platforms do not allow for near real-time monitoring of this macro-algae on beaches. This study proposes an innovative approach for monitoring Sargassum on beaches using Crowdsourcing for imagery collection, deep learning for automatic classification, and geographic information systems for visualizing the results. We have coined this collaborative process "Collective View". It offers a geotagged dataset of images illustrating the presence or absence of Sargassum on beaches located along the northern and eastern regions in the Yucatan Peninsula, in Mexico. This new dataset is the largest of its kind in surrounding areas. As part of the design process for Collective View, three convolutional neural networks (LeNet-5, AlexNet and VGG16) were modified and retrained to classify images, according to the presence or absence of Sargassum. Findings from this study revealed that AlexNet demonstrated the best performance, achieving a maximum recall of 94%. These results are good considering that the training was carried out using a relatively small set of unbalanced images. Finally, this study provides a first approach to mapping the Sargassum distribution along the beaches using the classified geotagged images and offers novel insight into how we can accurately map the arrival of algal blooms along the coastline.

NMR Metabolic Profiling of Sargassum Species Under Different Stabilization/Extraction Processes

The genus Sargassum is well represented by benthic and pelagic species, some of which form massive aggregates that can travel long distances due to the force of the ocean currents. Although they constitute an essential habitat for fish and invertebrate species, large accumulations of Sargassum in coastal areas generate several economic, environmental, and health impacts. It is important to recognize the species forming these aggregates, and identify the metabolites they produce, allowing for its exploitation, and therefore, better management practices. NMR metabolic profiling is a technique that can discriminate samples while detecting their unique or differential chemical features, and has been successfully used in the study and classification of several algal species. The present investigation studied the metabolic profiling of Sargassum species found on strandings at Puerto Morelos (Quintana Roo) east coast of the Mexican Caribbean. PCA of the 1 H-NMR profiles corresponding to S. natans, S. natans (morphotype VIII), S. fluitans, and a benthic Sargassum buxifolium allowed the discrimination of samples amongst them. Furthermore, discrimination between the two forms of S. natans was also possible. The PCA loading plot revealed that glutamine and glutamate have the highest influence in the clustering of the benthic Sargassum, while a high abundance of lactate, Myo-inositol, and trimethylamine is a unique feature from the S. natans morphotype VIII. Additional PLS-DA models showed that a heat-drying process improved the extraction of metabolites. Maceration and microwave-assisted extraction with water-ethanol led to similar profiles and thus any of them could be used in future investigations.

Instance Segmentation for Large, Multi-Channel Remote Sensing Imagery Using Mask-RCNN and a Mosaicking Approach

Instance segmentation is the state-of-the-art in object detection, and there are numerous applications in remote sensing data where these algorithms can produce significant results. Nevertheless, one of the main problems is that most algorithms use Red, Green, and Blue (RGB) images, whereas Satellite images often present more channels that can be crucial to improve performance. Therefore, the present work brings three contributions: (a) conversion system from ground truth polygon data into the Creating Common Object in Context (COCO) annotation format; (b) Detectron2 software source code adaptation and application on multi-channel imagery; and (c) large scene image mosaicking. We applied the procedure in a Center Pivot Irrigation System (CPIS) dataset with ground truth produced by the Brazilian National Water Agency (ANA) and Landsat-8 Operational Land Imager (OLI) imagery (7 channels with 30-m resolution). Center pivots are a modern irrigation system technique with massive growth potential in Brazil and other world areas. The round shapes with different textures, colors, and spectral behaviors make it appropriate to use Deep Learning instance segmentation. We trained the model using 512 × 512-pixel sized patches using seven different backbone structures (ResNet50- Feature Pyramid Network (FPN), Resnet50-DC5, ResNet50-C4, Resnet101-FPN, Resnet101-DC5, ResNet101-FPN, and ResNeXt101-FPN). The model evaluation used standard COCO metrics (Average Precision (AP), AP50, AP75, APsmall, APmedium, and AR100). ResNeXt101-FPN had the best results, with a 3% advantage over the second-best model (ResNet101-FPN). We also compared the ResNeXt101-FPN model in the seven-channel and RGB imagery, where the multi-channel model had a 3% advantage, demonstrating great improvement using a larger number of channels. This research is also the first with a mosaicking algorithm using instance segmentation models, where we tested in a 1536 × 1536-pixel image using a non-max suppression sorted by area method. The proposed methodology is innovative and suitable for many other remote sensing problems and medical imagery that often present more channels.

Deep-learning-based information mining from ocean remote-sensing imagery

With the continuous development of space and sensor technologies during the last 40 years, ocean remote sensing has entered into the big-data era with typical five-V (volume, variety, value, velocity and veracity) characteristics. Ocean remote-sensing data archives reach several tens of petabytes and massive satellite data are acquired worldwide daily. To precisely, efficiently and intelligently mine the useful information submerged in such ocean remote-sensing data sets is a big challenge. Deep learning-a powerful technology recently emerging in the machine-learning field-has demonstrated its more significant superiority over traditional physical- or statistical-based algorithms for image-information extraction in many industrial-field applications and starts to draw interest in ocean remote-sensing applications. In this review paper, we first systematically reviewed two deep-learning frameworks that carry out ocean remote-sensing-image classifications and then presented eight typical applications in ocean internal-wave/eddy/oil-spill/coastal-inundation/sea-ice/green-algae/ship/coral-reef mapping from different types of ocean remote-sensing imagery to show how effective these deep-learning frameworks are. Researchers can also readily modify these existing frameworks for information mining of other kinds of remote-sensing imagery.

Sargassum blooms in the Caribbean alter the trophic structure of the sea urchin Diadema antillarum

The arrival of large masses of drifting Sargassum since 2011 has caused changes in the natural dynamics of Caribbean coastal ecosystems. In the summer of 2015, unprecedented and massive mats of S. fluitans and S. natans have been observed throughout the Mexican Caribbean including exceptional accumulations ashore. This study uses stable isotopes to assess the impact of Sargassum blooms on the trophic dynamics of the Diadema antillarum sea urchin, a keystone herbivore on many Caribbean reefs. Bayesian models were used to estimate the variations in the relative proportions of carbon and nitrogen of assimilated algal resources. At three lagoon reef sites, the niche breadth of D. antillarum was analysed and compared under massive influx of drifting Sargassum spp. vs. no influx of Sargassum blooms. The effects of the leachates generated by the decomposition of Sargassum led to hypoxic conditions on these reefs and reduced the taxonomic diversity of macroalgal food sources available to D. antillarum. Our trophic data support the hypothesis that processes of assimilation of carbon and nitrogen were modified under Sargassum effect. Isotopic signatures of macroalgae associated with the reef sites exhibited significantly lower values of δ¹⁵N altering the natural herbivory of D. antillarum. The Stable Isotopes Analysis in R (SIAR) indicated that, under the influence of Sargassum blooms, certain algal resources (Dictyota, Halimeda and Udotea) were more assimilated due to a reduction in available algal resources. Despite being an abundant available resource, pelagic Sargassum was a negligible contributor to sea urchin diet. The Stable Isotope Bayesian Ellipses in R (SIBER) analysis displayed differences between sites, and suggests a reduction in trophic niche breadth, particularly in a protected reef lagoon. Our findings reveal that Sargassum blooms caused changes in trophic characteristics of D. antillarum with a negative impact by hypoxic conditions. These dynamics, coupled with the increase in organic matter in an oligotrophic system could lead to reduce coral reef ecosystem function.