Fully Convolutional Neural Network for Detection and Counting of Diatoms on Coatings after Short-Term Field Exposure

Efficient smartphone-based measurement of phosphorus in water

Agricultural runoff is one of the main sources of excess phosphorus (P) in different water bodies, subsequently leading to eutrophication and harmful algal blooms. To effectively monitor P levels in water, there is a need for simple measurement tools and extensive public involvement to enable regular and widespread sampling. Several smartphone-based P measurement methods have been reported, which extract red-green-blue (RGB) values from colorimetric reactions to build statistical regression models for P quantification. However, these methods typically require meticulous light conditions, involve initial equipment investment, and have undergone limited testing for large-scale applications. To overcome these limitations, this study developed a smartphone-based, equipment-free and facile P colorimetric analysis method. Following the standard procedure of the ascorbic acid approach, colorimetric reactions were captured by a smartphone camera, and RGB values were extracted using Python code for modeling. Different indoor light conditions, phone types, containers, and types of water samples were examined, resulting in a collection of 1922 images. The best regression model, employing random forest with RGB values and container types as inputs, achieved an R2 of 0.97 and an RMSE of 0.051 for P concentrations ranging from 0.01 to 1.0 mg P/L. Additionally, the optimal classification model could estimate the level of P below 0.1 mg P/L with an accuracy of 95.2 (or 77.4 % for <0.05 mg P/L). The strong performance of the developed models, which are also available freely online, suggests an easy and effective tool for more frequent P measurement and greater public involvement.

Microscopic image recognition of diatoms based on deep learning

Diatoms are a crucial component in the study of aquatic ecosystems and ancient environmental records. However, traditional methods for identifying diatoms, such as morphological taxonomy and molecular detection, are costly, are time consuming, and have limitations. To address these issues, we developed an extensive collection of diatom images, consisting of 7983 images from 160 genera and 1042 species, which we expanded to 49,843 through preprocessing, segmentation, and data augmentation. Our study compared the performance of different algorithms, including backbones, batch sizes, dynamic data augmentation, and static data augmentation on experimental results. We determined that the ResNet152 network outperformed other networks, producing the most accurate results with top-1 and top-5 accuracies of 85.97% and 95.26%, respectively, in identifying 1042 diatom species. Additionally, we propose a method that combines model prediction and cosine similarity to enhance the model's performance in low-probability predictions, achieving an 86.07% accuracy rate in diatom identification. Our research contributes significantly to the recognition and classification of diatom images and has potential applications in water quality assessment, ecological monitoring, and detecting changes in aquatic biodiversity.

Deep-Learning-Based Automated Tracking and Counting of Living Plankton in Natural Aquatic Environments

Plankton are widely distributed in the aquatic environment and serve as an indicator of water quality. Monitoring the spatiotemporal variation in plankton is an efficient approach to forewarning environmental risks. However, conventional microscopy counting is time-consuming and laborious, hindering the application of plankton statistics for environmental monitoring. In this work, an automated video-oriented plankton tracking workflow (AVPTW) based on deep learning is proposed for continuous monitoring of living plankton abundance in aquatic environments. With automatic video acquisition, background calibration, detection, tracking, correction, and statistics, various types of moving zooplankton and phytoplankton were counted at a time scale. The accuracy of AVPTW was validated with conventional counting via microscopy. Since AVPTW is only sensitive to mobile plankton, the temperature- and wastewater-discharge-induced plankton population variations were monitored online, demonstrating the sensitivity of AVPTW to environmental changes. The robustness of AVPTW was also confirmed with natural water samples from a contaminated river and an uncontaminated lake. Notably, automated workflows are essential for generating large amounts of data, which are a prerequisite for available data set construction and subsequent data mining. Furthermore, data-driven approaches based on deep learning pave a novel way for long-term online environmental monitoring and elucidating the correlation underlying environmental indicators. This work provides a replicable paradigm to combine imaging devices with deep-learning algorithms for environmental monitoring.

DiatomNet v1.0: A novel approach for automatic diatom testing for drowning diagnosis in forensically biomedical application

BACKGROUND AND OBJECTIVE

Diatom testing is supportive for drowning diagnosis in forensic medicine. However, it is very time-consuming and labor-intensive for technicians to identify microscopically a handful of diatoms in sample smears, especially under complex observable backgrounds. Recently, we successfully developed a software, named DiatomNet v1.0 intended to automatically identify diatom frustules in a whole slide under a clear background. Here, we introduced this new software and performed a validation study to elucidate how DiatomNet v1.0 improved its performance with the influence of visible impurities.

METHODS

DiatomNet v1.0 has an intuitive, user-friendly and easy-to-learn graphical user interface (GUI) built in the Drupal and its core architecture for slide analysis including a convolutional neural network (CNN) is written in Python language. The build-in CNN model was evaluated for diatom identification under very complex observable backgrounds with mixtures of common impurities, including carbon pigments and sand sediments. Compared to the original model, the enhanced model following optimization with limited new datasets was evaluated systematically by independent testing and random control trials (RCTs).

RESULTS

In independent testing, the original DiatomNet v1.0 was moderately affected, especially when higher densities of impurities existed, and achieved a low recall of 0.817 and F1 score of 0.858 but good precision of 0.905. Following transfer learning with limited new datasets, the enhanced version had better results, with recall and F1 score values of 0.968. A comparative study on real slides showed that the upgraded DiatomNet v1.0 obtained F1 scores of 0.86 and 0.84 for carbon pigment and sand sediment, respectively, slightly worse than manual identification (carbon pigment: 0.91; sand sediment: 0.86), but much less time was needed.

CONCLUSIONS

The study verified that forensic diatom testing with aid of DiatomNet v1.0 is much more efficient than traditionally manual identification even under complex observable backgrounds. In terms of forensic diatom testing, we proposed a suggested standard on build-in model optimization and evaluation to strengthen the software's generalization in potentially complex conditions.

Improving deep learning-based segmentation of diatoms in gigapixel-sized virtual slides by object-based tile positioning and object integrity constraint

Diatoms represent one of the morphologically and taxonomically most diverse groups of microscopic eukaryotes. Light microscopy-based taxonomic identification and enumeration of frustules, the silica shells of these microalgae, is broadly used in aquatic ecology and biomonitoring. One key step in emerging digital variants of such investigations is segmentation, a task that has been addressed before, but usually in manually captured megapixel-sized images of individual diatom cells with a mostly clean background. In this paper, we applied deep learning-based segmentation methods to gigapixel-sized, high-resolution scans of diatom slides with a realistically cluttered background. This setup requires large slide scans to be subdivided into small images (tiles) to apply a segmentation model to them. This subdivision (tiling), when done using a sliding window approach, often leads to cropping relevant objects at the boundaries of individual tiles. We hypothesized that in the case of diatom analysis, reducing the amount of such cropped objects in the training data can improve segmentation performance by allowing for a better discrimination of relevant, intact frustules or valves from small diatom fragments, which are considered irrelevant when counting diatoms. We tested this hypothesis by comparing a standard sliding window / fixed-stride tiling approach with two new approaches we term object-based tile positioning with and without object integrity constraint. With all three tiling approaches, we trained Mask-R-CNN and U-Net models with different amounts of training data and compared their performance. Object-based tiling with object integrity constraint led to an improvement in pixel-based precision by 12-17 percentage points without substantially impairing recall when compared with standard sliding window tiling. We thus propose that training segmentation models with object-based tiling schemes can improve diatom segmentation from large gigapixel-sized images but could potentially also be relevant for other image domains.

Semantic segmentation for fully automated macrofouling analysis on coatings after field exposure

Biofouling is a major challenge for sustainable shipping, filter membranes, heat exchangers, and medical devices. The development of fouling-resistant coatings requires the evaluation of their effectiveness. Such an evaluation is usually based on the assessment of fouling progression after different exposure times to the target medium (e.g. salt water). The manual assessment of macrofouling requires expert knowledge about local fouling communities due to high variances in phenotypical appearance, has single-image sampling inaccuracies for certain species, and lacks spatial information. Here an approach for automatic image-based macrofouling analysis was presented. A dataset with dense labels prepared from field panel images was made and a convolutional network (adapted U-Net) for the semantic segmentation of different macrofouling classes was proposed. The establishment of macrofouling localization allows for the generation of a successional model which enables the determination of direct surface attachment and in-depth epibiotic studies.

An improved automated diatom detection method based on YOLOv5 framework and its preliminary study for taxonomy recognition in the forensic diatom test

The diatom test is a forensic technique that can provide supportive evidence in the diagnosis of drowning but requires the laborious observation and counting of diatoms using a microscopy with too much effort, and therefore it is promising to introduce artificial intelligence (AI) to make the test process automatic. In this article, we propose an artificial intelligence solution based on the YOLOv5 framework for the automatic detection and recognition of the diatom genera. To evaluate the performance of this AI solution in different scenarios, we collected five lab-grown diatom genera and samples of some organic tissues from drowning cases to investigate the potential upper/lower limits of the capability in detecting the diatoms and recognizing their genera. Based on the study of the article, a recall score of 0.95 together with the corresponding precision score of 0.9 were achieved on the samples of the five lab-grown diatom genera via cross-validation, and the accuracy of the evaluation in the cases of kidney and liver is above 0.85 based on the precision and recall scores, which demonstrate the effectiveness of the AI solution to be used in drowning forensic routine.

Machine learning in natural and engineered water systems

Water resources of desired quality and quantity are the foundation for human survival and sustainable development. To better protect the water environment and conserve water resources, efficient water management, purification, and transportation are of critical importance. In recent years, machine learning (ML) has exhibited its practicability, reliability, and high efficiency in numerous applications; furthermore, it has solved conventional and emerging problems in both natural and engineered water systems. For example, ML can predict various water quality indicators in situ and real-time by considering the complex interactions among water-related variables. ML approaches can also solve emerging pollution problems with proven rules or universal mechanisms summarized from the related research. Moreover, by applying image recognition technology to analyze the relationships between image information and physicochemical properties of the research object, ML can effectively identify and characterize specific contaminants. In view of the bright prospects of ML, this review comprehensively summarizes the development of ML applications in natural and engineered water systems. First, the concept and modeling steps of ML are briefly introduced, including data preparation, algorithm selection and model evaluation. In addition, comprehensive applications of ML in recent studies, including predicting water quality, mapping groundwater contaminants, classifying water resources, tracing contaminant sources, and evaluating pollutant toxicity in natural water systems, as well as modeling treatment techniques, assisting characterization analysis, purifying and distributing drinking water, and collecting and treating sewage water in engineered water systems, are summarized. Finally, the advantages and disadvantages of commonly used algorithms are analyzed according to their structures and mechanisms, and recommendations on the selection of ML algorithms for different studies, as well as prospects on the application and development of ML in water science are proposed. This review provides references for solving a wider range of water-related problems and brings further insights into the intelligent development of water science.

Identification of diatom taxonomy by a combination of region-based full convolutional network, online hard example mining, and shape priors of diatoms

Diatom test is one of the commonly used diagnostic methods for drowning in forensic pathology, which provides supportive evidence for drowning. However, in forensic practice, it is time-consuming and laborious for forensic experts to classify and count diatoms, whereas artificial intelligence (AI) is superior to human experts in processing data and carrying out classification tasks. Some AI techniques have focused on searching diatoms and classifying diatoms. But, they either could not classify diatoms correctly or were time-consuming. Conventional detection deep network has been used to overcome these problems but failed to detect the occluded diatoms and the diatoms similar to the background heavily, which could lead to false positives or false negatives. In order to figure out the problems above, an improved region-based full convolutional network (R-FCN) with online hard example mining and the shape prior of diatoms was proposed. The online hard example mining (OHEM) was coupled with the R-FCN to boost the capacity of detecting the occluded diatoms and the diatoms similar to the background heavily and the priors of the shape of the common diatoms were explored and introduced to the anchor generation strategy of the region proposal network in the R-FCN to locate the diatoms precisely. The results showed that the proposed approach significantly outperforms several state-of-the-art methods and could detect the diatom precisely without missing the occluded diatoms and the diatoms similar to the background heavily. From the study, we could conclude that (1) the proposed model can locate the position and identify the genera of common diatoms more accurately; (2) this method can reduce the false positives or false negatives in forensic practice; and (3) it is a time-saving method and can be introduced.

Versatile Microfluidic Platform for Automated Live-Cell Hyperspectral Imaging Applied to Cold Climate Cyanobacterial Biofilms

Microfluidic bioanalytical platforms are driving discoveries from synthetic biology to the health sciences. In this work, we present a platform for in vivo live-cell imaging and automated species detection in mixed cyanobacterial biofilms from cold climate environments. Using a multimodal microscope with custom optics applied to a chip with six parallel growth channels, we monitored biofilm dynamics via continuous imaging at natural irradiance levels. Machine learning algorithms were applied to the collected hyperspectral images for automatic segmentation of mixed-species biofilms into individual species of cyanobacteria with similar filamentous morphology. The coupling of microfluidic technology with modern multimodal imaging and computer vision systems provides a versatile platform for the study of cause-and-effect scenarios of cyanobacterial biofilms, which are important elements of many ecosystems, including lakes and rivers of the polar regions.