Underwater digital holography for studies of marine plankton

Confocal LiDAR for remote high-resolution imaging of auto-fluorescence in aquatic media

Spatially resolved in situ monitoring of plankton can provide insights on the impacts of climate change on aquatic ecosystems due to their vital role in the biological carbon pump. However, high-resolution underwater imaging is technically complex and restricted to small close-range volumes with current techniques. Here, we report a novel inelastic scanning confocal light detection and ranging (LiDAR) system for remote underwater volumetric imaging of fluorescent objects. A continuous wave excitation beam is combined with a pinhole in a conjugated detection plane to reject out-of-focus scattering and accomplish near-diffraction limited probe volumes. The combination of bi-directional scanning with remote focusing enables the acquisition of three-dimensional data. We experimentally determine the point spread and axial weighting functions, and demonstrate selective volumetric imaging of obstructed layers through spatial filtering. Finally, we spatially resolve in vivo autofluorescence from sub-millimeter Acocyclops royi copepods to demonstrate the applicability of our novel instrument in non-intrusive morphological and spectroscopic studies of aquatic fauna. The proposed system constitutes a unique tool e.g. for profiling chlorophyll distributions and for quantitative studies of zooplankton with reduced interference from intervening scatterers in the water column that degrade the the performance of conventional imaging systems currently in place.

In Situ Measurements of Plankton Biorhythms Using Submersible Holographic Camera

The paper presents a diagnostic complex for plankton studies using the miniDHC (digital holographic camera). Its capabilities to study the rhythmic processes in plankton ecosystems were demonstrated using the natural testing in Lake Baikal in summer. The results of in situ measurements of plankton to detect the synchronization of collective biological rhythms with medium parameters are presented and interpreted. The most significant rhythms in terms of the correlation of their parameters with medium factors are identified. The study shows that the correlation with water temperature at the mooring site has the greatest significance and reliability. The results are verified with biodiversity data obtained by the traditional mesh method. The experience and results of the study can be used for the construction of a stationary station to monitor the ecological state of the water area through the digitalization of plankton behavior.

Combining multi-marker metabarcoding and digital holography to describe eukaryotic plankton across the Newfoundland Shelf

The planktonic diversity throughout the oceans is vital to ecosystem functioning and linked to environmental change. Plankton monitoring tools have advanced considerably with high-throughput in-situ digital cameras and genomic sequencing, opening new challenges for high-frequency observations of community composition, structure, and species discovery. Here, we combine multi-marker metabarcoding based on nuclear 18S (V4) and plastidial 16S (V4–V5) rRNA gene amplicons with a digital in-line holographic microscope to provide a synoptic diversity survey of eukaryotic plankton along the Newfoundland Shelf (Canada) during the winter transition phase of the North Atlantic bloom phenomenon. Metabarcoding revealed a rich eukaryotic diversity unidentifiable in the imaging samples, confirming the presence of ecologically important saprophytic protists which were unclassifiable in matching images, and detecting important groups unobserved or taxonomically unresolved during similar sequencing campaigns in the Northwest Atlantic Ocean. In turn, imaging analysis provided quantitative observations of widely prevalent plankton from every trophic level. Despite contrasting plankton compositions portrayed by each sampling method, both capture broad spatial differences between the northern and southern sectors of the Newfoundland Shelf and suggest complementary estimations of important features in eukaryotic assemblages. Future tasks will involve standardizing digital imaging and metabarcoding for wider use and consistent, comparable ocean observations.

Machine Learning for the Study of Plankton and Marine Snow from Images

Quantitative imaging instruments produce a large number of images of plankton and marine snow, acquired in a controlled manner, from which the visual characteristics of individual objects and their in situ concentrations can be computed. To exploit this wealth of information, machine learning is necessary to automate tasks such as taxonomic classification. Through a review of the literature, we highlight the progress of those machine classifiers and what they can and still cannot be trusted for. Several examples showcase how the combination of quantitative imaging with machine learning has brought insights on pelagic ecology. They also highlight what is still missing and how images could be exploited further through trait-based approaches. In the future, we suggest deeper interactions with the computer sciences community, the adoption of data standards, and the more systematic sharing of databases to build a global community of pelagic image providers and users.

Phototropic response features for different systematic groups of mesoplankton under adverse environmental conditions

Current trends in the application of bioindication methods are related to the use of submersible tools that perform real-time measurements directly in the studied aquatic environment. The methods based on the registration of changes in the behavioral responses of zooplankton, in particular Crustaceans, which make up the vast majority of the biomass in water areas, seem quite promising. However, the multispecies composition of natural planktonic biocenoses poses the need to consider the potential difference in the sensitivity of organisms to pollutants. This paper describes laboratory studies of the phototropic response of plankton to attracting light. The studies were carried out on a model natural community that in equal amounts includes Daphnia magna, Daphnia pulex, and Cyclops vicinus, as well as on the monoculture groups of these species. The phototropic response was initiated by the attracting light with a wavelength of 532 nm close to the local maximum of the reflection spectrum of chlorella microalgae. Standard potassium bichromate was used as the model pollutant. The largest phototropic response value is registered in the assemblage. The concentration growth rate of crustaceans in the illuminated volume was 4.5 ± 0.3 ind (L min)-1. Of the studied species, the phototropic response was mostly expressed in Daphnia magna (3.7 ± 0.4 ind (L min)-1), while in Daphnia pulex, it was reduced to 2.4 ± 0.2 ind (L min)-1, and in Cyclops vicinus, it was very small-0.16 ± 0.02 ind (L min)-1. This is caused by peculiar trophic behavior of phyto- and zoophages. The addition of a pollutant, namely potassium bichromate, caused a decrease in the concentration rate of crustaceans in the attracting light zone, while a dose-dependent change in phototropic responses was observed in a group of species and the Daphnia magna assemblage. The results of laboratory studies showed high potential of using the phototropic response of zooplankton to monitor the quality of its habitat thus ensuring the early diagnostics of water pollution. Besides, the paper shows the possibility of quantifying the phototropic response of zooplankton using submersible digital holographic cameras (DHC).

Unsupervised feature learning and clustering of particles imaged in raw holograms using an autoencoder

Digital holography is a useful tool to image microscopic particles. Reconstructed holograms give high-resolution shape information that can be used to identify the types of particles. However, the process of reconstructing holograms is computationally intensive and cannot easily keep up with the rate of data acquisition on low-power sensor platforms. In this work, we explore the possibility of performing object clustering on holograms that have not been reconstructed, i.e., images of raw interference patterns, using the latent representations of a deep-learning autoencoder and a self-organizing mapping network in a fully unsupervised manner. We demonstrate this concept on synthetically generated holograms of different shapes, where clustering of raw holograms achieves an accuracy of 94.4%. This is comparable to the 97.4% accuracy achieved using the reconstructed holograms of the same targets. Directly clustering raw holograms takes less than 0.1 s per image using a low-power CPU board. This represents a three-order of magnitude reduction in processing time compared to clustering of reconstructed holograms and makes it possible to interpret targets in real time on low-power sensor platforms. Experiments on real holograms demonstrate significant gains in clustering accuracy through the use of synthetic holograms to train models. Clustering accuracy increased from 47.1% when the models were trained only on the real raw holograms, to 64.1% when the models were entirely trained on the synthetic raw holograms, and further increased to 75.9% when models were trained on the both synthetic and real datasets using transfer learning. These results are broadly comparable to those achieved when reconstructed holograms are used, where the highest accuracy of 70% achieved when clustering raw holograms outperforms the highest accuracy achieved when clustering reconstructed holograms by a significant margin for our datasets.

Plankton reconstruction through robust statistical optical tomography

Plankton interact with the environment according to their size and three-dimensional (3D) structure. To study them outdoors, these translucent specimens are imaged in situ. Light projects through a specimen in each image. The specimen has a random scale, drawn from the population's size distribution and random unknown pose. The specimen appears only once before drifting away. We achieve 3D tomography using such a random ensemble to statistically estimate an average volumetric distribution of the plankton type and specimen size. To counter errors due to non-rigid deformations, we weight the data, drawing from advanced models developed for cryo-electron microscopy. The weights convey the confidence in the quality of each datum. This confidence relies on a statistical error model. We demonstrate the approach on live plankton using an underwater field microscope.

Underwater Holographic Sensor for Plankton Studies In Situ including Accompanying Measurements

The paper presents an underwater holographic sensor to study marine particles—a miniDHC digital holographic camera, which may be used as part of a hydrobiological probe for accompanying (background) measurements. The results of field measurements of plankton are given and interpreted, their verification is performed. Errors of measurements and classification of plankton particles are estimated. MiniDHC allows measurement of the following set of background data, which is confirmed by field tests: plankton concentration, average size and size dispersion of individuals, particle size distribution, including on major taxa, as well as water turbidity and suspension statistics. Version of constructing measuring systems based on modern carriers of operational oceanography for the purpose of ecological diagnostics of the world ocean using autochthonous plankton are discussed. The results of field measurements of plankton using miniDHC as part of a hydrobiological probe are presented and interpreted, and their verification is carried out. The results of comparing the data on the concentration of individual taxa obtained using miniDHC with the data obtained by the traditional method using plankton catching with a net showed a difference of no more than 23%. The article also contains recommendations for expanding the potential of miniDHC, its purpose indicators, and improving metrological characteristics.

Plankton classification with high-throughput submersible holographic microscopy and transfer learning

BACKGROUND

Plankton are foundational to marine food webs and an important feature for characterizing ocean health. Recent developments in quantitative imaging devices provide in-flow high-throughput sampling from bulk volumes-opening new ecological challenges exploring microbial eukaryotic variation and diversity, alongside technical hurdles to automate classification from large datasets. However, a limited number of deployable imaging instruments have been coupled with the most prominent classification algorithms-effectively limiting the extraction of curated observations from field deployments. Holography offers relatively simple coherent microscopy designs with non-intrusive 3-D image information, and rapid frame rates that support data-driven plankton imaging tasks. Classification benchmarks across different domains have been set with transfer learning approaches, focused on repurposing pre-trained, state-of-the-art deep learning models as classifiers to learn new image features without protracted model training times. Combining the data production of holography, digital image processing, and computer vision could improve in-situ monitoring of plankton communities and contribute to sampling the diversity of microbial eukaryotes.

RESULTS

Here we use a light and portable digital in-line holographic microscope (The HoloSea) with maximum optical resolution of 1.5 μm, intensity-based object detection through a volume, and four different pre-trained convolutional neural networks to classify > 3800 micro-mesoplankton (> 20 μm) images across 19 classes. The maximum classifier performance was quickly achieved for each convolutional neural network during training and reached F1-scores > 89%. Taking classification further, we show that off-the-shelf classifiers perform strongly across every decision threshold for ranking a majority of the plankton classes.

CONCLUSION

These results show compelling baselines for classifying holographic plankton images, both rare and plentiful, including several dinoflagellate and diatom groups. These results also support a broader potential for deployable holographic microscopes to sample diverse microbial eukaryotic communities, and its use for high-throughput plankton monitoring.

Identification of microplastics in a large water volume by integrated holography and Raman spectroscopy

A noncontact method to identify sparsely distributed plastic pellets is proposed by integrating holography and Raman spectroscopy in this study. Polystyrene and poly(methyl methacrylate) resin pellets with a size of 3 mm located in a 20 cm water channel were illuminated using a collimated continuous wave laser beam with a diameter of 4 mm and wavelength of 785 nm. The same laser beam was used to take a holographic image and Raman spectrum of a pellet to identify the shape, size, and composition of material. Using the compact system, the morphological and chemical analysis of pellets in a large volume of water was performed. The reported method demonstrates the potential for noncontact continuous in situ monitoring of microplastics in water without collection and separation.

Autofocusing and image fusion for multi-focus plankton imaging by digital holographic microscopy

Digital holographic microscopy is becoming increasingly useful for the analysis of marine plankton. In this study, we investigate autofocusing and image fusion in digital holographic microscopy. We propose an area metric autofocusing method and an improved wavelet-based image fusion method. In the area metric autofocusing method, a hologram image is initially segmented into several plankton regions for focus plane detection, and an area metric is then applied to these regions. In the improved wavelet-based image fusion method, a marked map is introduced for labeling each plankton region with the order of refocus plane images that accounts for the most pixels. The results indicate that the area metric autofocusing method applied to each plankton region provides a higher depth resolution accuracy than a number of general autofocusing methods, and the mean accuracy increases by approximately 33%. The improved wavelet-based image fusion method can fuse more than nine reconstructed plane images at a time and effectively eliminate fringes and speckle noise, and the fused image is much clearer than that of a general wavelet-based method, a sparse decomposition method, and a pulse-coupled neural networks method. This work has practical value for plankton imaging using digital holographic microscopy.

Investigation of autofluorescence in zooplankton for use in classification of larval salmon lice

We present autofluorescence of six zooplankton species, including salmon lice (Lepeophtheirus salmonis), for the purpose of classification in marine environments. Using a 410 nm excitation wavelength, we find that all measured zooplankton species exhibit broad cyan fluorescence centered around 510-520 nm. Furthermore, salmon lice show an absence of red fluorescence from undigested chlorophyll, which is measured from the gut of the herbivorous zooplankton species. We show the capability to distinguish noneating species, including salmon lice, from algae-eating species using a dual-band analysis of the fluorescence spectra. This shows the potential of autofluorescence as an important signature in real-time monitoring and classification of salmon lice.

Fast autofocusing in digital holography using the magnitude differential

Typical methods of automatic estimation of focusing in digital holography calculate every single reconstructed frame to get a critical function and then ascertain the focal plane by finding the extreme value of that function. Here, we propose a digital holographic autofocusing method that computes the focused distance using the first longitudinal difference of the magnitude of the reconstructed image. We demonstrate the proposed method with both numerical simulations and optical experiments of amplitude-contrast and phase-contrast objects. The results suggest that the proposed method performs better than other existing methods, in terms of applicability and computation efficiency, with potential applications in industrial and biomedical inspections where automatic focus tracking is necessary.

Quantitative phase imaging of arthropods

Classification of arthropods is performed by characterization of fine features such as setae and cuticles. An unstained whole arthropod specimen mounted on a slide can be preserved for many decades, but is difficult to study since current methods require sample manipulation or tedious image processing. Spatial light interference microscopy (SLIM) is a quantitative phase imaging (QPI) technique that is an add-on module to a commercial phase contrast microscope. We use SLIM to image a whole organism springtail Ceratophysella denticulata mounted on a slide. This is the first time, to our knowledge, that an entire organism has been imaged using QPI. We also demonstrate the ability of SLIM to image fine structures in addition to providing quantitative data that cannot be obtained by traditional bright field microscopy.

A Mach-Zender digital holographic microscope with sub-micrometer resolution for imaging and tracking of marine micro-organisms

Digital holographic microscopy is an ideal tool for investigation of microbial motility. However, most designs do not exhibit sufficient spatial resolution for imaging bacteria. In this study we present an off-axis Mach-Zehnder design of a holographic microscope with spatial resolution of better than 800 nm and the ability to resolve bacterial samples at varying densities over a 380 μm × 380 μm × 600 μm three-dimensional field of view. Larger organisms, such as protozoa, can be resolved in detail, including cilia and flagella. The instrument design and performance are presented, including images and tracks of bacterial and protozoal mixed samples and pure cultures of six selected species. Organisms as small as 1 μm (bacterial spores) and as large as 60 μm (Paramecium bursaria) may be resolved and tracked without changes in the instrument configuration. Finally, we present a dilution series investigating the maximum cell density that can be imaged, a type of analysis that has not been presented in previous holographic microscopy studies.

James Clerk Maxwell 150 years on

This paper is the preface to a special Issue of Phil. Trans. R. Soc. A reporting selected proceedings of the international conference marking the 150th anniversary of James Clerk Maxwell's professorial debut at Marischal College, Aberdeen. Following an introduction to Marischal College, a brief historical note summarizes Maxwell's life prior to his entering the college as professor of natural philosophy. The preface provides a short summary of the event and overviews the contributed papers devoted to subjects covering a wide range of Maxwell's research interests and their modern developments. The mixture of review and research papers reflects both the fundamental importance and the diverse applicability of Maxwell's works in electromagnetics, colour science, dynamics and kinetics. Acknowledgements are given to the individuals and bodies who made the conference the success that it was.