Underwater Hyperspectral Imaging Technology and Its Applications for Detecting and Mapping the Seafloor: A Review

High-speed scanless entire bandwidth mid-infrared chemical imaging

Mid-infrared spectroscopy probes molecular vibrations to identify chemical species and functional groups. Therefore, mid-infrared hyperspectral imaging is one of the most powerful and promising candidates for chemical imaging using optical methods. Yet high-speed and entire bandwidth mid-infrared hyperspectral imaging has not been realized. Here we report a mid-infrared hyperspectral chemical imaging technique that uses chirped pulse upconversion of sub-cycle pulses at the image plane. This technique offers a lateral resolution of 15 µm, and the field of view is adjustable between 800 µm × 600 µm to 12 mm × 9 mm. The hyperspectral imaging produces a 640 × 480 pixel image in 8 s, which covers a spectral range of 640-3015 cm^-1, comprising 1069 wavelength points and offering a wavenumber resolution of 2.6-3.7 cm^-1. For discrete frequency mid-infrared imaging, the measurement speed reaches a frame rate of 5 kHz, the repetition rate of the laser. As a demonstration, we effectively identified and mapped different components in a microfluidic device, plant cell, and mouse embryo section. The great capacity and latent force of this technique in chemical imaging promise to be applied to many fields such as chemical analysis, biology, and medicine.

Semantic Segmentation of Hyperspectral Remote Sensing Images Based on PSE-UNet Model

With the development of deep learning, the use of convolutional neural networks (CNN) to improve the land cover classification accuracy of hyperspectral remote sensing images (HSRSI) has become a research hotspot. In HSRSI semantics segmentation, the traditional dataset partition method may cause information leakage, which poses challenges for a fair comparison between models. The performance of the model based on "convolutional-pooling-fully connected" structure is limited by small sample sizes and high dimensions of HSRSI. Moreover, most current studies did not involve how to choose the number of principal components with the application of the principal component analysis (PCA) to reduce dimensionality. To overcome the above challenges, firstly, the non-overlapping sliding window strategy combined with the judgment mechanism is introduced, used to split the hyperspectral dataset. Then, a PSE-UNet model for HSRSI semantic segmentation is designed by combining PCA, the attention mechanism, and UNet, and the factors affecting the performance of PSE-UNet are analyzed. Finally, the cumulative variance contribution rate (CVCR) is introduced as a dimensionality reduction metric of PCA to study the Hughes phenomenon. The experimental results with the Salinas dataset show that the PSE-UNet is superior to other semantic segmentation algorithms and the results can provide a reference for HSRSI semantic segmentation.

A Tunable Hyperspectral Imager for Detection and Quantification of Marine Biofouling on Coated Surfaces

Fouling control coatings (FCCs) are used to prevent the accumulation of marine biofouling on, e.g., ship hulls, which causes increased fuel consumption and the global spread of non-indigenous species. The standards for performance evaluations of FCCs rely on visual inspections, which induce a degree of subjectivity. The use of RGB images for objective evaluations has already received interest from several authors, but the limited acquired information restricts detailed analyses class-wise. This study demonstrates that hyperspectral imaging (HSI) expands the specificity of biofouling assessments of FCCs by capturing distinguishing spectral features. We developed a staring-type hyperspectral imager using a liquid crystal tunable filter as the wavelength selective element. A novel light-emitting diode illumination system with high and uniform irradiance was designed to compensate for the low-filter transmittance. A spectral library was created from reflectance-calibrated optical signatures of representative biofouling species and coated panels. We trained a neural network on the annotated library to assign a class to each pixel. The model was evaluated on an artificially generated target, and global accuracy of 95% was estimated. The classifier was tested on coated panels (exposed at the CoaST Maritime Test Centre) with visible intergrown biofouling. The segmentation results were used to determine the coverage percentage per class. Although a detailed taxonomic description might be complex due to spectral similarities among groups, these results demonstrate the feasibility of HSI for repeatable and quantifiable biofouling detection on coated surfaces.

Exploring time series of hyperspectral images for cold water coral stress response analysis

Hyperspectral imaging (HSI) is a promising technology for environmental monitoring with a lot of undeveloped potential due to the high dimensionality and complexity of the data. If temporal effects are studied, such as in a monitoring context, the analysis becomes more challenging as time is added to the dimensions of space (image coordinates) and wavelengths. We conducted a series of laboratory experiments to investigate the impact of different stressor exposure patterns on the spectrum of the cold water coral Desmophyllum pertusum. 65 coral samples were divided into 12 groups, each group being exposed to different types and levels of particles. Hyperspectral images of the coral samples were collected at four time points from prior to exposure to 6 weeks after exposure. To investigate the relationships between the corals’ spectral signatures and controlled experimental parameters, a new software tool for interactive visual exploration was developed and applied, the HypIX (Hyperspectral Image eXplorer) web tool. HypIX combines principles from exploratory data analysis, information visualization and machine learning-based dimension reduction. This combination enables users to select regions of interest (ROI) in all dimensions (2D space, time point and spectrum) for a flexible integrated inspection. We propose two HypIX workflows to find relationships in time series of hyperspectral datasets, namely morphology-based filtering workflow and embedded driven response analysis workflow. With these HypIX workflows three users identified different temporal and spatial patterns in the spectrum of corals exposed to different particle stressor conditions. Corals exposed to particles tended to have a larger change rate than control corals, which was evident as a shifted spectrum. The responses, however, were not uniform for coral samples undergoing the same exposure treatments, indicating individual tolerance levels. We also observed a good inter-observer agreement between the three HyPIX users, indicating that the proposed workflow can be applied to obtain reproducible HSI analysis results.

Underwater Hyperspectral Imaging of Arctic Macroalgal Habitats during the Polar Night Using a Novel Mini-ROV-UHI Portable System

We describe an Underwater Hyperspectral Imager (UHI) deployed on an instrument-carrying platform consisting of two interconnected mini-ROVs (Remotely Operated Vehicle) for the mapping and monitoring of Arctic macroalgal habitats in Kongsfjorden (Svalbard) during the Polar Night. The mini-ROV-UHI system is easy to transport, assemble and deploy from shore, even under the dark, icy and cold conditions of the Arctic Polar Night. The system can be operated by two persons, keeping the operational costs low. In vivo hyperspectral reflectance of collected specimens of brown, red and green macroalgae was measured with a spectrometer in the lab to provide a spectral library for supervised pigment group classification based on UHI photomosaics. The in situ UHI-photomosaics provided detailed information of the areal coverage of the seafloor substrate (16%), as well as brown (51% habitat cover), red (18%), and green (14%) macroalgae, with spatial resolution in the range of cm and spectral resolution of 2 nm. The collected specimens from the mapped area were also used for species identification and health state evaluation. This innovative UHI sampling method provides significant information about macroalgal distribution and physiology, and due to its flexibility in terms of deployment, it is applicable to a variety of environments.

The Color Improvement of Underwater Images Based on Light Source and Detector

As one of the most direct approaches to perceive the world, optical images can provide plenty of useful information for underwater applications. However, underwater images often present color deviation due to the light attenuation in the water, which reduces the efficiency and accuracy in underwater applications. To improve the color reproduction of underwater images, we proposed a method with adjusting the spectral component of the light source and the spectral response of the detector. Then, we built the experimental setup to study the color deviation of underwater images with different lamps and different cameras. The experimental results showed that, a) in terms of light source, the color deviation of an underwater image with warm light LED (Light Emitting Diode) (with the value of Δa*2+Δb*2 being 26.58) was the smallest compared with other lamps, b) in terms of detectors, the color deviation of images with the 3×CMOS RGB camera (a novel underwater camera with three CMOS sensors developed for suppressing the color deviation in our team) (with the value of Δa*2+Δb*2 being 25.25) was the smallest compared with other cameras. The experimental result (i.e., the result of color improvement between different lamps or between different cameras) verified our assumption that the underwater image color could be improved by adjusting the spectral component of the light source and the spectral response of the detector. Differing from the color improvement method with image processing, this color-improvement method was based on hardware, which had advantages, including more image information being retained and less-time being consumed.

Classifying Crop Types Using Two Generations of Hyperspectral Sensors (Hyperion and DESIS) with Machine Learning on the Cloud

Advances in spaceborne hyperspectral (HS) remote sensing, cloud-computing, and machine learning can help measure, model, map and monitor agricultural crops to address global food and water security issues, such as by providing accurate estimates of crop area and yield to model agricultural productivity. Leveraging these advances, we used the Earth Observing-1 (EO-1) Hyperion historical archive and the new generation DLR Earth Sensing Imaging Spectrometer (DESIS) data to evaluate the performance of hyperspectral narrowbands in classifying major agricultural crops of the U.S. with machine learning (ML) on Google Earth Engine (GEE). EO-1 Hyperion images from the 2010–2013 growing seasons and DESIS images from the 2019 growing season were used to classify three world crops (corn, soybean, and winter wheat) along with other crops and non-crops near Ponca City, Oklahoma, USA. The supervised classification algorithms: Random Forest (RF), Support Vector Machine (SVM), and Naive Bayes (NB), and the unsupervised clustering algorithm WekaXMeans (WXM) were run using selected optimal Hyperion and DESIS HS narrowbands (HNBs). RF and SVM returned the highest overall producer’s, and user’s accuracies, with the performances of NB and WXM being substantially lower. The best accuracies were achieved with two or three images throughout the growing season, especially a combination of an earlier month (June or July) and a later month (August or September). The narrow 2.55 nm bandwidth of DESIS provided numerous spectral features along the 400–1000 nm spectral range relative to smoother Hyperion spectral signatures with 10 nm bandwidth in the 400–2500 nm spectral range. Out of 235 DESIS HNBs, 29 were deemed optimal for agricultural study. Advances in ML and cloud-computing can greatly facilitate HS data analysis, especially as more HS datasets, tools, and algorithms become available on the Cloud.

Bag of Features (BoF) Based Deep Learning Framework for Bleached Corals Detection

Coral reefs are the sub-aqueous calcium carbonate structures collected by the invertebrates known as corals. The charm and beauty of coral reefs attract tourists, and they play a vital role in preserving biodiversity, ceasing coastal erosion, and promoting business trade. However, they are declining because of over-exploitation, damaging fishery, marine pollution, and global climate changes. Also, coral reefs help treat human immune-deficiency virus (HIV), heart disease, and coastal erosion. The corals of Australia’s great barrier reef have started bleaching due to the ocean acidification, and global warming, which is an alarming threat to the earth’s ecosystem. Many techniques have been developed to address such issues. However, each method has a limitation due to the low resolution of images, diverse weather conditions, etc. In this paper, we propose a bag of features (BoF) based approach that can detect and localize the bleached corals before the safety measures are applied. The dataset contains images of bleached and unbleached corals, and various kernels are used to support the vector machine so that extracted features can be classified. The accuracy of handcrafted descriptors and deep convolutional neural networks is analyzed and provided in detail with comparison to the current method. Various handcrafted descriptors like local binary pattern, a histogram of an oriented gradient, locally encoded transform feature histogram, gray level co-occurrence matrix, and completed joint scale local binary pattern are used for feature extraction. Specific deep convolutional neural networks such as AlexNet, GoogLeNet, VGG-19, ResNet-50, Inception v3, and CoralNet are being used for feature extraction. From experimental analysis and results, the proposed technique outperforms in comparison to the current state-of-the-art methods. The proposed technique achieves 99.08% accuracy with a classification error of 0.92%. A novel bleached coral positioning algorithm is also proposed to locate bleached corals in the coral reef images.