Remote Hyperspectral Imaging Acquisition and Characterization for Marine Litter Detection

The use of artificial intelligence algorithms to detect macroplastics in aquatic environments: A critical review

The presence of macroplastic (MP) is having serious consequences on natural ecosystems, directly affecting biota and human wellbeing. Given this scenario, estimating MPs' abundance is crucial for assessing the issue and formulating effective waste management strategies. In this context, the main objective of this critical review is to analyze the use of machine learning (ML) techniques, with a particular interest in deep learning (DL) approaches, to detect, classify and quantify MPs in aquatic environments, supported by datasets such as satellite or aerial images and video recordings taken by unmanned aerial vehicles. This article provides a concise overview of artificial intelligence concepts, followed by a bibliometric analysis and a critical review. The search methodology aimed to categorize the scientific contributions through temporal and spatial criteria for bibliometric analysis, whereas the critical review was based on generating homogeneous groups according to the complexity of ML and DL methods, as well as the type of dataset. In light of the review carried out, classical ML techniques, such as random forest or support vector machines, showed robustness in MPs detection. However, it seems that achieving optimal efficiencies in multiclass classification is a limitation for these methods. Consequently, more advanced techniques such as DL approaches are taking the lead for the detection and multiclass classification of MPs. A series of architectures based on convolutional neural networks, and the use of complex pre-trained models through the transfer learning, are currently being explored (e.g., VGG16 and YOLO models), although currently the computational expense is high due to the need for processing large volumes of data. Additionally, there seems to be a trend towards detecting smaller plastic, which need higher resolution images. Finally, it is important to stress that since 2020 there has been a significant increase in scientific research focusing on transformer-based architectures for object detection. Although this can be considered the current state of the art, no studies have been identified that utilize these architectures for MP detection.

Endmember extraction and abundance estimation algorithm based on double-compressed sampling

Based on double-compressed sampling, a hyperspectral spectral unmixing algorithm (SU_DCS) is proposed, which could directly complete the endmember extraction and abundance estimation. On the basis of the linear mixed model (LMM), we designed spatial and spectral sampling matrices, obtained spatial and spectral measurement data, and constructed a joint unmixing model containing endmember and abundance information. By using operator separation and Lagrangian multiplier algorithm, the endmember matrix, abundance matrix and remixing image can be quickly obtained by matrix operation. The parameters of the unmixing algorithm, including regularization parameter, convergence threshold and spatial sampling rate, are determined using synthetic simulated hyperspectral data. The proposed algorithm is applied to two kinds of real hyperspectral data, with or without ground truth, in order to verify the effectiveness and reliability of the algorithm. Firstly, we provide the performance of the algorithm on real datasets without ground truth. Compared with algorithm VCA_FCLS and algorithm CPPCA_VCA_FCLS, the endmember spectral curve extracted by the proposed SU_DCS is almost consistent with that obtained by VCA_FCLS, and is more smooth than that of obtained by CPPCA_VCA_FCLS. Additionally, the abundance estimation map estimated by the SU_DCS has consistency with the results obtained by VCA_FCLS. Moreover, the proposed SU_DCS has higher peak signal-to-noise ratio (PSNR) for remixing images with higher computational efficiency. Secondly, we provide the performance of the proposed algorithm on four real datasets with ground truth, including dataset Cuprite, dataset Samson, dataset Jasper and dataset Urban. We provide the results of endmember extraction and abundance estimation from the compressed data under different sampling rate conditions. The extracted endmember maintains good consistency with the true spectral curves, and the estimated abundance map can also maintain good spatial consistency with the ground truth. The comparison results with other four comparative algorithms also indicate that the proposed algorithm can obtain relatively accurate endmembers and abundance information from compressed data, the reliability and validity of the proposed algorithm have been proved. In summary, the main innovation of the proposed algorithm is that it can extract endmembers and estimate abundance with high accuracy from a small amount of measurement data.

Eliminating the interference of water for direct sensing of submerged plastics using hyperspectral near-infrared imager

Interference from water in the reflectance spectra of plastics is a major obstacle to optical sensing of plastics in aquatic environments. Here we present evidence of the feasibility of sensing plastics in water using hyperspectral near-infrared to shortwave-infrared imaging techniques. We captured hyperspectral images of nine polymers submerged to four depths (2.5-15 mm) in water using a hyperspectral imaging system that utilizes near-infrared to shortwave-infrared light sources. We also developed algorithms to predict the reflectance spectra of each polymer in water using the spectra of the dry plastics and water as independent variables in a multiple linear regression model after a logarithmic transformation. A narrow 1100-1300 nm wavelength range was advantageous for detection of polyethylene, polystyrene, and polyvinyl chloride in water down to the 160-320 µm size range, while a wider 970-1670 nm wavelength range was beneficial for polypropylene reflectance spectrum prediction in water. Furthermore, we found that the spectra of the other five polymers, comprising polycarbonate, acrylonitrile butadiene styrene, phenol formaldehyde, polyacetal, and polymethyl methacrylate, could also be predicted within their respective optimized wavelength ranges. Our findings provide fundamental information for direct sensing of plastics in water on both benchtop and airborne platforms.

Airborne Spectral Reflectance Dataset of Submerged Plastic Targets in a Coastal Environment

Among the emerging applications of remote sensing technologies, the remote detection of plastic litter has observed successful applications in recent years. However, while the number of studies and datasets for spectral characterization of plastic is growing, few studies address plastic litter while being submerged in natural seawater in an outdoor context. This study aims to investigate the feasibility of hyperspectral characterization of submerged plastic litter in less-than-ideal conditions. We present a hyperspectral dataset of eight different polymers in field conditions, taken by an unmanned aerial vehicle (UAV) on different days in a three-week period. The measurements were carried out off the coast of Mytilene, Greece. The team collected the dataset using a Bayspec OCI-F push broom sensor from 25 m and 40 m height above the water. For a contextual background, the dataset also contains optical (RGB) high-resolution orthomosaics.

Aerial and underwater drones for marine litter monitoring in shallow coastal waters: factors influencing item detection and cost-efficiency

Although marine litter monitoring has increased over the years, the pollution of coastal waters is still understudied and there is a need for spatial and temporal data. Aerial (UAV) and underwater (ROV) drones have demonstrated their potential as monitoring tools at coastal sites; however, suitable conditions for use and cost-efficiency of the methods still need attention. This study tested UAVs and ROVs for the monitoring of floating, submerged, and seafloor items using artificial plastic plates and assessed the influence of water conditions (water transparency, color, depth, bottom substrate), item characteristics (color and size), and method settings (flight/dive height) on detection accuracy. A cost-efficiency analysis suggests that both UAV and ROV methods lie within the same cost and efficiency category as current on-boat observation and scuba diving methods and shall be considered for further testing in real scenarios for official marine litter monitoring methods.

Marine litter and climate change: Inextricably connected threats to the world's oceans

The global issues of climate change and marine litter are interlinked and understanding these connections is key to managing their combined risks to marine biodiversity and ultimately society. For example, fossil fuel-based plastics cause direct emissions of greenhouse gases and therefore are an important contributing factor to climate change, while other impacts of plastics can manifest as alterations in key species and habitats in coastal and marine environments. Marine litter is acknowledged as a threat multiplier that acts with other stressors such as climate change to cause far greater damage than if they occurred in isolation. On the other hand, while climate change can lead to increased inputs of litter into the marine environment, the presence of marine litter can also undermine the climate resilience of marine ecosystems. There is increasing evidence that that climate change and marine litter are inextricably linked, although these interactions and the resulting effects vary widely across oceanic regions and depend on the particular characteristics of specific marine environments. Ecosystem resilience approaches, that integrate climate change with other local stressors, offer a suitable framework to incorporate the consideration of marine litter where that is deemed to be a risk, and to steer, coordinate and prioritise research and monitoring, as well as management, policy, planning and action to effectively tackle the combined risks and impacts from climate change and marine litter.

Close-range remote sensing-based detection and identification of macroplastics on water assisted by artificial intelligence: A review

Detection and identification of macroplastic debris in aquatic environments is crucial to understand and counter the growing emergence and current developments in distribution and deposition of macroplastics. In this context, close-range remote sensing approaches revealing spatial and spectral properties of macroplastics are very beneficial. To date, field surveys and visual census approaches are broadly acknowledged methods to acquire information, but since 2018 techniques based on remote sensing and artificial intelligence are advancing. Despite their proven efficiency, speed and wide applicability, there are still obstacles to overcome, especially when looking at the availability and accessibility of data. Thus, our review summarizes state-of-the-art research about the visual recognition and identification of different sorts of macroplastics. The focus is on both data acquisition techniques and evaluation methods, including Machine Learning and Deep Learning, but resulting products and published data will also be taken into account. Our aim is to provide a critical overview and outlook in a time where this research direction is thriving fast. This study shows that most Machine Learning and Deep Learning approaches are still in an infancy state regarding accuracy and detail when compared to visual monitoring, even though their results look very promising.

Detection and Classification of Floating Plastic Litter Using a Vessel-Mounted Video Camera and Deep Learning

Marine plastic pollution is a major environmental concern, with significant ecological, economic, public health and aesthetic consequences. Despite this, the quantity and distribution of marine plastics is poorly understood. Better understanding of the global abundance and distribution of marine plastic debris is vital for global mitigation and policy. Remote sensing methods could provide substantial data to overcome this issue. However, developments have been hampered by the limited availability of in situ data, which are necessary for development and validation of remote sensing methods. Current in situ methods of floating macroplastics (size greater than 1 cm) are usually conducted through human visual surveys, often being costly, time-intensive and limited in coverage. To overcome this issue, we present a novel approach to collecting in situ data using a trained object-detection algorithm to detect and quantify marine macroplastics from video footage taken from vessel-mounted general consumer cameras. Our model was able to successfully detect the presence or absence of plastics from real-world footage with an accuracy of 95.2% without the need to pre-screen the images for horizon or other landscape features, making it highly portable to other environmental conditions. Additionally, the model was able to differentiate between plastic object types with a Mean Average Precision of 68% and an F1-Score of 0.64. Further analysis suggests that a way to improve the separation among object types using only object detection might be through increasing the proportion of the image area covered by the plastic object. Overall, these results demonstrate how low-cost vessel-mounted cameras combined with machine learning have the potential to provide substantial harmonised in situ data of global macroplastic abundance and distribution.

Using a UAV Thermal Infrared Camera for Monitoring Floating Marine Plastic Litter

In recent years, the remote sensing of marine plastic litter has been rapidly evolving and the technology is most advanced in the visible (VIS), near-infrared (NIR), and short-wave infrared (SWIR) wavelengths. It has become clear that sensing using VIS-SWIR bands, based on the surface reflectance of sunlight, would benefit from complementary measurements using different technologies. Thermal infrared (TIR) sensing shows potential as a novel method for monitoring macro plastic litter floating on the water surface, as the physics behind surface-leaving TIR is different. We assessed a thermal radiance model for floating plastic litter using a small UAV-grade FLIR Vue Pro R 640 thermal camera by flying it over controlled floating plastic litter targets during the day and night and in different seasons. Experiments in the laboratory supported the field measurements. We investigated the effects of environmental conditions, such as temperatures, light intensity, the presence of clouds, and biofouling. TIR sensing could complement observations from VIS, NIR, and SWIR in several valuable ways. For example, TIR sensing could be used for monitoring during the night, to detect plastics invisible to VIS-SWIR, to discriminate whitecaps from marine litter, and to detect litter pollution over clear, shallow waters. In this study, we have shown the previously unconfirmed potential of using TIR sensing for monitoring floating plastic litter.

Review on the Application of Hyperspectral Imaging Technology of the Exposed Cortex in Cerebral Surgery

The study of brain science is vital to human health. The application of hyperspectral imaging in biomedical fields has grown dramatically in recent years due to their unique optical imaging method and multidimensional information acquisition. Hyperspectral imaging technology can acquire two-dimensional spatial information and one-dimensional spectral information of biological samples simultaneously, covering the ultraviolet, visible and infrared spectral ranges with high spectral resolution, which can provide diagnostic information about the physiological, morphological and biochemical components of tissues and organs. This technology also presents finer spectral features for brain imaging studies, and further provides more auxiliary information for cerebral disease research. This paper reviews the recent advance of hyperspectral imaging in cerebral diagnosis. Firstly, the experimental setup, image acquisition and pre-processing, and analysis methods of hyperspectral technology were introduced. Secondly, the latest research progress and applications of hyperspectral imaging in brain tissue metabolism, hemodynamics, and brain cancer diagnosis in recent years were summarized briefly. Finally, the limitations of the application of hyperspectral imaging in cerebral disease diagnosis field were analyzed, and the future development direction was proposed.

Hyperspectral Anomaly Detection Based on Wasserstein Distance and Spatial Filtering

Since anomaly targets in hyperspectral images (HSIs) with high spatial resolution appear as connected areas instead of single pixels or subpixels, both spatial and spectral information of HSIs can be exploited for a hyperspectal anomaly detection (AD) task. This article proposes a hyperspectral AD method based on Wasserstein distance (WD) and spatial filtering (called AD-WDSF). Based on the assumption that both background and anomaly targets obey the multivariate Gaussian distribution, background and anomaly target distributions are estimated in the local regions of HSIs. Subsequently, the anomaly intensity of test pixels centered in the local regions are determined via measuring the WD between background and anomaly target distributions. Lastly, spatial filters, i.e., guided filter (GF), total variation curvature filter (TVCF), and Maxtree filter, are exploited to further refine detection results. Experimental results conducted on two real hyperspectral data sets demonstrate that the proposed method achieves competitive detection performance compared with the state-of-the-art AD methods.

Attention Mechanism and Depthwise Separable Convolution Aided 3DCNN for Hyperspectral Remote Sensing Image Classification

Hyperspectral Remote Rensing Image (HRSI) classification based on Convolution Neural Network (CNN) has become one of the hot topics in the field of remote sensing. However, the high dimensional information and limited training samples are prone to the Hughes phenomenon for hyperspectral remote sensing images. Meanwhile, high-dimensional information processing also consumes significant time and computing power, or the extracted features may not be representative, resulting in unsatisfactory classification efficiency and accuracy. To solve these problems, an attention mechanism and depthwise separable convolution are introduced to the three-dimensional convolutional neural network (3DCNN). Thus, 3DCNN-AM and 3DCNN-AM-DSC are proposed for HRSI classification. Firstly, three hyperspectral datasets (Indian pines, University of Pavia and University of Houston) are used to analyze the patchsize and dataset allocation ratio (Training set: Validation set: Test Set) in the performance of 3DCNN and 3DCNN-AM. Secondly, in order to improve work efficiency, principal component analysis (PCA) and autoencoder (AE) dimension reduction methods are applied to reduce data dimensionality, and maximize the classification accuracy of the 3DCNN, but it will still take time. Furthermore, the HRSI classification model 3DCNN-AM and 3DCNN-AM-DSC are applied to classify with the three classic HRSI datasets. Lastly, the classification accuracy index and time consumption are evaluated. The results indicate that 3DCNN-AM could improve classification accuracy and reduce computing time with the dimension reduction dataset, and the 3DCNN-AM-DSC model can reduce the training time by a maximum of 91.77% without greatly reducing the classification accuracy. The results of the three classic hyperspectral datasets illustrate that 3DCNN-AM-DSC can improve the classification performance and reduce the time required for model training. It may be a new way to tackle hyperspectral datasets in HRSl classification tasks without dimensionality reduction.

Beached and Floating Litter Surveys by Unmanned Aerial Vehicles: Operational Analogies and Differences

The abundance of litter pollution in the marine environment has been increasing globally. Remote sensing techniques are valuable tools to advance knowledge on litter abundance, distribution and dynamics. Images collected by Unmanned Aerial Vehicles (UAV, aka drones) are highly efficient to map and monitor local beached (BL) and floating (FL) marine litter items. In this work, the operational insights to carry out both BL and FL surveys using UAVs are detailly described. In particular, flight planning and deployment, along with image products processing and analysis, are reported and compared. Furthermore, analogies and differences between UAV-based BL and FL mapping are discussed, with focus on the challenges related to BL and FL item detection and recognition. Given the efficiency of UAV to map BL and FL, this remote sensing technique can replace traditional methods for litter monitoring, further improving the knowledge of marine litter dynamics in the marine environment. This communication aims at helping researchers in planning and performing optimized drone-based BL and FL surveys.

Litter Detection with Deep Learning: A Comparative Study

Pollution in the form of litter in the natural environment is one of the great challenges of our times. Automated litter detection can help assess waste occurrences in the environment. Different machine learning solutions have been explored to develop litter detection tools, thereby supporting research, citizen science, and volunteer clean-up initiatives. However, to the best of our knowledge, no work has investigated the performance of state-of-the-art deep learning object detection approaches in the context of litter detection. In particular, no studies have focused on the assessment of those methods aiming their use in devices with low processing capabilities, e.g., mobile phones, typically employed in citizen science activities. In this paper, we fill this literature gap. We performed a comparative study involving state-of-the-art CNN architectures (e.g., Faster RCNN, Mask-RCNN, EfficientDet, RetinaNet and YOLO-v5), two litter image datasets and a smartphone. We also introduce a new dataset for litter detection, named PlastOPol, composed of 2418 images and 5300 annotations. The experimental results demonstrate that object detectors based on the YOLO family are promising for the construction of litter detection solutions, with superior performance in terms of detection accuracy, processing time, and memory footprint.

UAV Approach for Detecting Plastic Marine Debris on the Beach: A Case Study in the Po River Delta (Italy)

Anthropogenic marine debris (AMD) represent a global threat for aquatic environments. It is important to locate and monitor the distribution and presence of macroplastics along beaches to prevent degradation into microplastics (MP), which are potentially more harmful and more difficult to remove. UAV imaging represents a quick method for acquiring pictures with a ground spatial resolution of a few centimeters. In this work, we investigate strategies for AMD mapping on beaches with different ground resolutions and with elevation and multispectral data in support of RGB orthomosaics. Operators with varying levels of expertise and knowledge of the coastal environment map the AMD on four to five transects manually, using a range of photogrammetric tools. The initial survey was repeated after one year; in both surveys, beach litter was collected and further analyzed in the laboratory. Operators assign three levels of confidence when recognizing and describing AMD. Preliminary validation of results shows that items identified with high confidence were almost always classified properly. Approaching the detected items in terms of surface instead of a simple count increased the percentage of mapped litter significantly when compared to those collected. Multispectral data in near-infrared (NIR) wavelengths and digital surface models (DSMs) did not significantly improve the efficiency of manual mapping, even if vegetation features were removed using NDVI maps. In conclusion, this research shows that a good solution for performing beach AMD mapping can be represented by using RGB imagery with a spatial resolution of about 200 pix/m for detecting macroplastics and, in particular, focusing on the largest items. From the point of view of assessing and monitoring potential sources of MP, this approach is not only feasible but also quick, practical, and sustainable.

Quantifying Floating Plastic Debris at Sea Using Vessel-Based Optical Data and Artificial Intelligence

Despite recent advances in remote sensing of large accumulations of floating plastic debris, mainly in coastal regions, the quantification of individual macroplastic objects (>50 cm) remains challenging. Here, we have trained an object-detection algorithm by selecting and labeling footage of floating plastic debris recorded offshore with GPS-enabled action cameras aboard vessels of opportunity. Macroplastic numerical concentrations are estimated by combining the object detection solution with bulk processing of the optical data. Our results are consistent with macroplastic densities predicted by global plastic dispersal models, and reveal first insights into how camera recorded offshore macroplastic densities compare to micro- and mesoplastic concentrations collected with neuston trawls.