Automated Diatom Classification (Part A): Handcrafted Feature Approaches

A lightweight convolutional neural network (CNN) model for diatom classification: DiatomNet

Diatoms are a type of algae with many species. Accurate and quick classification of diatom species is important in many fields, such as water quality analysis and weather change forecasting. Traditional methods for diatom classification, specifically morphological taxonomy and molecular detection, are time-consuming and may not provide satisfactory performance. However, in recent years, deep learning has demonstrated impressive performance in this task, just like other image classification problems. On the other hand, networks with more layers do not guarantee increased accuracy. While increasing depth can be useful in capturing complex features and patterns, it also introduces challenges such as vanishing gradients, overfitting, and optimization challenges. Therefore, in our work, we propose DiatomNet, a lightweight convolutional neural network (CNN) model that can classify diatom species accurately while requiring low computing resources. A recently introduced dataset consisting of 3,027 diatom images and 68 diatom species is used to train and evaluate the model. The model is compared with well-known and successful CNN models (i.e., AlexNet, GoogleNet, Inceptionv3, ResNet18, VGG16, and Xception) and their customized versions obtained with transfer learning. The comparison is based on several success metrics: accuracy, precision, recall, F-measure, number of learnable parameters, training, and prediction time. Eventually, the experimental results reveal that DiatomNet outperforms the other models regarding all metrics with just a few exceptions. Therefore, it is a lightweight but strong candidate for diatom classification tasks.

Microscopic image recognition of diatoms based on deep learning

Diatoms are a crucial component in the study of aquatic ecosystems and ancient environmental records. However, traditional methods for identifying diatoms, such as morphological taxonomy and molecular detection, are costly, are time consuming, and have limitations. To address these issues, we developed an extensive collection of diatom images, consisting of 7983 images from 160 genera and 1042 species, which we expanded to 49,843 through preprocessing, segmentation, and data augmentation. Our study compared the performance of different algorithms, including backbones, batch sizes, dynamic data augmentation, and static data augmentation on experimental results. We determined that the ResNet152 network outperformed other networks, producing the most accurate results with top-1 and top-5 accuracies of 85.97% and 95.26%, respectively, in identifying 1042 diatom species. Additionally, we propose a method that combines model prediction and cosine similarity to enhance the model's performance in low-probability predictions, achieving an 86.07% accuracy rate in diatom identification. Our research contributes significantly to the recognition and classification of diatom images and has potential applications in water quality assessment, ecological monitoring, and detecting changes in aquatic biodiversity.

Improving deep learning-based segmentation of diatoms in gigapixel-sized virtual slides by object-based tile positioning and object integrity constraint

Diatoms represent one of the morphologically and taxonomically most diverse groups of microscopic eukaryotes. Light microscopy-based taxonomic identification and enumeration of frustules, the silica shells of these microalgae, is broadly used in aquatic ecology and biomonitoring. One key step in emerging digital variants of such investigations is segmentation, a task that has been addressed before, but usually in manually captured megapixel-sized images of individual diatom cells with a mostly clean background. In this paper, we applied deep learning-based segmentation methods to gigapixel-sized, high-resolution scans of diatom slides with a realistically cluttered background. This setup requires large slide scans to be subdivided into small images (tiles) to apply a segmentation model to them. This subdivision (tiling), when done using a sliding window approach, often leads to cropping relevant objects at the boundaries of individual tiles. We hypothesized that in the case of diatom analysis, reducing the amount of such cropped objects in the training data can improve segmentation performance by allowing for a better discrimination of relevant, intact frustules or valves from small diatom fragments, which are considered irrelevant when counting diatoms. We tested this hypothesis by comparing a standard sliding window / fixed-stride tiling approach with two new approaches we term object-based tile positioning with and without object integrity constraint. With all three tiling approaches, we trained Mask-R-CNN and U-Net models with different amounts of training data and compared their performance. Object-based tiling with object integrity constraint led to an improvement in pixel-based precision by 12-17 percentage points without substantially impairing recall when compared with standard sliding window tiling. We thus propose that training segmentation models with object-based tiling schemes can improve diatom segmentation from large gigapixel-sized images but could potentially also be relevant for other image domains.

An improved automated diatom detection method based on YOLOv5 framework and its preliminary study for taxonomy recognition in the forensic diatom test

The diatom test is a forensic technique that can provide supportive evidence in the diagnosis of drowning but requires the laborious observation and counting of diatoms using a microscopy with too much effort, and therefore it is promising to introduce artificial intelligence (AI) to make the test process automatic. In this article, we propose an artificial intelligence solution based on the YOLOv5 framework for the automatic detection and recognition of the diatom genera. To evaluate the performance of this AI solution in different scenarios, we collected five lab-grown diatom genera and samples of some organic tissues from drowning cases to investigate the potential upper/lower limits of the capability in detecting the diatoms and recognizing their genera. Based on the study of the article, a recall score of 0.95 together with the corresponding precision score of 0.9 were achieved on the samples of the five lab-grown diatom genera via cross-validation, and the accuracy of the evaluation in the cases of kidney and liver is above 0.85 based on the precision and recall scores, which demonstrate the effectiveness of the AI solution to be used in drowning forensic routine.

Identification of diatom taxonomy by a combination of region-based full convolutional network, online hard example mining, and shape priors of diatoms

Diatom test is one of the commonly used diagnostic methods for drowning in forensic pathology, which provides supportive evidence for drowning. However, in forensic practice, it is time-consuming and laborious for forensic experts to classify and count diatoms, whereas artificial intelligence (AI) is superior to human experts in processing data and carrying out classification tasks. Some AI techniques have focused on searching diatoms and classifying diatoms. But, they either could not classify diatoms correctly or were time-consuming. Conventional detection deep network has been used to overcome these problems but failed to detect the occluded diatoms and the diatoms similar to the background heavily, which could lead to false positives or false negatives. In order to figure out the problems above, an improved region-based full convolutional network (R-FCN) with online hard example mining and the shape prior of diatoms was proposed. The online hard example mining (OHEM) was coupled with the R-FCN to boost the capacity of detecting the occluded diatoms and the diatoms similar to the background heavily and the priors of the shape of the common diatoms were explored and introduced to the anchor generation strategy of the region proposal network in the R-FCN to locate the diatoms precisely. The results showed that the proposed approach significantly outperforms several state-of-the-art methods and could detect the diatom precisely without missing the occluded diatoms and the diatoms similar to the background heavily. From the study, we could conclude that (1) the proposed model can locate the position and identify the genera of common diatoms more accurately; (2) this method can reduce the false positives or false negatives in forensic practice; and (3) it is a time-saving method and can be introduced.

Automatic Detection of Freshwater Phytoplankton Specimens in Conventional Microscopy Images

Water safety and quality can be compromised by the proliferation of toxin-producing phytoplankton species, requiring continuous monitoring of water sources. This analysis involves the identification and counting of these species which requires broad experience and knowledge. The automatization of these tasks is highly desirable as it would release the experts from tedious work, eliminate subjective factors, and improve repeatability. Thus, in this preliminary work, we propose to advance towards an automatic methodology for phytoplankton analysis in digital images of water samples acquired using regular microscopes. In particular, we propose a novel and fully automatic method to detect and segment the existent phytoplankton specimens in these images using classical computer vision algorithms. The proposed method is able to correctly detect sparse colonies as single phytoplankton candidates, thanks to a novel fusion algorithm, and is able to differentiate phytoplankton specimens from other image objects in the microscope samples (like minerals, bubbles or detritus) using a machine learning based approach that exploits texture and colour features. Our preliminary experiments demonstrate that the proposed method provides satisfactory and accurate results.

Learning Diatoms Classification from a Dry Test Slide by Holographic Microscopy

Diatoms are among the dominant phytoplankters in marine and freshwater habitats, and important biomarkers of water quality, making their identification and classification one of the current challenges for environmental monitoring. To date, taxonomy of the species populating a water column is still conducted by marine biologists on the basis of their own experience. On the other hand, deep learning is recognized as the elective technique for solving image classification problems. However, a large amount of training data is usually needed, thus requiring the synthetic enlargement of the dataset through data augmentation. In the case of microalgae, the large variety of species that populate the marine environments makes it arduous to perform an exhaustive training that considers all the possible classes. However, commercial test slides containing one diatom element per class fixed in between two glasses are available on the market. These are usually prepared by expert diatomists for taxonomy purposes, thus constituting libraries of the populations that can be found in oceans. Here we show that such test slides are very useful for training accurate deep Convolutional Neural Networks (CNNs). We demonstrate the successful classification of diatoms based on a proper CNNs ensemble and a fully augmented dataset, i.e., creation starting from one single image per class available from a commercial glass slide containing 50 fixed species in a dry setting. This approach avoids the time-consuming steps of water sampling and labeling by skilled marine biologists. To accomplish this goal, we exploit the holographic imaging modality, which permits the accessing of a quantitative phase-contrast maps and a posteriori flexible refocusing due to its intrinsic 3D imaging capability. The network model is then validated by using holographic recordings of live diatoms imaged in water samples i.e., in their natural wet environmental condition.

Deep learning-based diatom taxonomy on virtual slides

Deep convolutional neural networks are emerging as the state of the art method for supervised classification of images also in the context of taxonomic identification. Different morphologies and imaging technologies applied across organismal groups lead to highly specific image domains, which need customization of deep learning solutions. Here we provide an example using deep convolutional neural networks (CNNs) for taxonomic identification of the morphologically diverse microalgal group of diatoms. Using a combination of high-resolution slide scanning microscopy, web-based collaborative image annotation and diatom-tailored image analysis, we assembled a diatom image database from two Southern Ocean expeditions. We use these data to investigate the effect of CNN architecture, background masking, data set size and possible concept drift upon image classification performance. Surprisingly, VGG16, a relatively old network architecture, showed the best performance and generalizing ability on our images. Different from a previous study, we found that background masking slightly improved performance. In general, training only a classifier on top of convolutional layers pre-trained on extensive, but not domain-specific image data showed surprisingly high performance (F1 scores around 97%) with already relatively few (100–300) examples per class, indicating that domain adaptation to a novel taxonomic group can be feasible with a limited investment of effort.

A Low-Cost Automated Digital Microscopy Platform for Automatic Identification of Diatoms

Currently, microalgae (i.e., diatoms) constitute a generally accepted bioindicator of water quality and therefore provide an index of the status of biological ecosystems. Diatom detection for specimen counting and sample classification are two difficult time-consuming tasks for the few existing expert diatomists. To mitigate this challenge, in this work, we propose a fully operative low-cost automated microscope, integrating algorithms for: (1) stage and focus control, (2) image acquisition (slide scanning, stitching, contrast enhancement), and (3) diatom detection and a prospective specimen classification (among 80 taxa). Deep learning algorithms have been applied to overcome the difficult selection of image descriptors imposed by classical machine learning strategies. With respect to the mentioned strategies, the best results were obtained by deep neural networks with a maximum precision of 86% (with the YOLO network) for detection and 99.51% for classification, among 80 different species (with the AlexNet network). All the developed operational modules are integrated and controlled by the user from the developed graphical user interface running in the main controller. With the developed operative platform, it is noteworthy that this work provides a quite useful toolbox for phycologists in their daily challenging tasks to identify and classify diatoms.

Fully Convolutional Neural Network for Detection and Counting of Diatoms on Coatings after Short-Term Field Exposure

While the use of deep learning is a valuable technology for automatic detection systems for medical data and images, the biofouling community is still lacking an analytical tool for the detection and counting of diatoms on samples after short-term field exposure. In this work, a fully convolutional neural network was implemented as a fast and simple approach to detect diatoms on two-channel (fluorescence and phase-contrast) microscopy images by predicting bounding boxes. The developed approach performs well with only a small number of trainable parameters and a F1 score of 0.82. Counting diatoms was evaluated on a data set of 600 microscopy images of three different surface chemistries (hydrophilic and hydrophobic) and is very similar to counting by humans while demanding only a fraction of the analysis time.

Automated diatom searching in the digital scanning electron microscopy images of drowning cases using the deep neural networks

Forensic diatom test has been widely accepted as a way of providing supportive evidences in the diagnosis of drowning. The current workflow is primarily based on the observation of diatoms by forensic pathologists under a microscopy, and this process can be very time-consuming. In this paper, we demonstrate a deep learning-based approach for automatically searching diatoms in scanning electron microscopic images. Cross-validation studies were performed to evaluate the influence of magnification on performance. Moreover, various training strategies were tested to improve the performance of detection. The conclusion shows that our approach can satisfy the necessary requirements to be integrated as part of an automatic forensic diatom test.

Identification to species level of live single microalgal cells from plankton samples with matrix-free laser/desorption ionization mass spectrometry

INTRODUCTION

Marine planktonic communities are complex microbial consortia often dominated by microscopic algae. The taxonomic identification of individual phytoplankton cells usually relies on their morphology and demands expert knowledge. Recently, a live single-cell mass spectrometry (LSC-MS) pipeline was developed to generate metabolic profiles of microalgae.

OBJECTIVE

Taxonomic identification of diverse microalgal single cells from collection strains and plankton samples based on the metabolic fingerprints analyzed with matrix-free laser desorption/ionization high-resolution mass spectrometry.

METHODS

Matrix-free atmospheric pressure laser-desorption ionization mass spectrometry was performed to acquire single-cell mass spectra from collection strains and prior identified environmental isolates. The computational identification of microalgal species was performed by spectral pattern matching (SPM). Three similarity scores and a bootstrap-derived confidence score were evaluated in terms of their classification performance. The effects of high and low-mass resolutions on the classification success were evaluated.

RESULTS

Several hundred single-cell mass spectra from nine genera and nine species of marine microalgae were obtained. SPM enabled the identification of single cells at the genus and species level with high accuracies. The receiver operating characteristic (ROC) curves indicated a good performance of the similarity measures but were outperformed by the bootstrap-derived confidence scores.

CONCLUSION

This is the first study to solve taxonomic identification of microalgae based on the metabolic fingerprints of the individual cell using an SPM approach.

Diatom identification including life cycle stages through morphological and texture descriptors

Diatoms are unicellular algae present almost wherever there is water. Diatom identification has many applications in different fields of study, such as ecology, forensic science, etc. In environmental studies, algae can be used as a natural water quality indicator. The diatom life cycle consists of the set of stages that pass through the successive generations of each species from the initial to the senescent cells. Life cycle modeling is a complex process since in general the distribution of the parameter vectors that represent the variations that occur in this process is non-linear and of high dimensionality. In this paper, we propose to characterize the diatom life cycle by the main features that change during the algae life cycle, mainly the contour shape and the texture. Elliptical Fourier Descriptors (EFD) are used to describe the diatom contour while phase congruency and Gabor filters describe the inner ornamentation of the algae. The proposed method has been tested with a small algae dataset (eight different classes and more than 50 samples per type) using supervised and non-supervised classification techniques obtaining accuracy results up to 99% and 98% respectively.

Quantitative comparison of taxa and taxon concepts in the diatom genus Fragilariopsis: a case study on using slide scanning, multiexpert image annotation, and image analysis in taxonomy1

Semiautomated methods for microscopic image acquisition, image analysis, and taxonomic identification have repeatedly received attention in diatom analysis. Less well studied is the question whether and how such methods might prove useful for clarifying the delimitation of species that are difficult to separate for human taxonomists. To try to answer this question, three very similar Fragilariopsis species endemic to the Southern Ocean were targeted in this study: F. obliquecostata, F. ritscheri, and F. sublinearis. A set of 501 extended focus depth specimen images were obtained using a standardized, semiautomated microscopic procedure. Twelve diatomists independently identified these specimen images in order to reconcile taxonomic opinions and agree upon a taxonomic gold standard. Using image analyses, we then extracted morphometric features representing taxonomic characters of the target taxa. The discriminating ability of individual morphometric features was tested visually and statistically, and multivariate classification experiments were performed to test the agreement of the quantitatively defined taxa assignments with expert consensus opinion. Beyond an updated differential diagnosis of the studied taxa, our study also shows that automated imaging and image analysis procedures for diatoms are coming close to reaching a broad applicability for routine use.

Low-cost oblique illumination: an image quality assessment

We study the effectiveness of several low-cost oblique illumination filters to improve overall image quality, in comparison with standard bright field imaging. For this purpose, a dataset composed of 3360 diatom images belonging to 21 taxa was acquired. Subjective and objective image quality assessments were done. The subjective evaluation was performed by a group of diatom experts by psychophysical test where resolution, focus, and contrast were assessed. Moreover, some objective nonreference image quality metrics were applied to the same image dataset to complete the study, together with the calculation of several texture features to analyze the effect of these filters in terms of textural properties. Both image quality evaluation methods, subjective and objective, showed better results for images acquired using these illumination filters in comparison with the no filtered image. These promising results confirm that this kind of illumination filters can be a practical way to improve the image quality, thanks to the simple and low cost of the design and manufacturing process.