1
|
Gunduz H, Gunal S. A lightweight convolutional neural network (CNN) model for diatom classification: DiatomNet. PeerJ Comput Sci 2024; 10:e1970. [PMID: 38660184 PMCID: PMC11042002 DOI: 10.7717/peerj-cs.1970] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/16/2023] [Accepted: 03/11/2024] [Indexed: 04/26/2024]
Abstract
Diatoms are a type of algae with many species. Accurate and quick classification of diatom species is important in many fields, such as water quality analysis and weather change forecasting. Traditional methods for diatom classification, specifically morphological taxonomy and molecular detection, are time-consuming and may not provide satisfactory performance. However, in recent years, deep learning has demonstrated impressive performance in this task, just like other image classification problems. On the other hand, networks with more layers do not guarantee increased accuracy. While increasing depth can be useful in capturing complex features and patterns, it also introduces challenges such as vanishing gradients, overfitting, and optimization challenges. Therefore, in our work, we propose DiatomNet, a lightweight convolutional neural network (CNN) model that can classify diatom species accurately while requiring low computing resources. A recently introduced dataset consisting of 3,027 diatom images and 68 diatom species is used to train and evaluate the model. The model is compared with well-known and successful CNN models (i.e., AlexNet, GoogleNet, Inceptionv3, ResNet18, VGG16, and Xception) and their customized versions obtained with transfer learning. The comparison is based on several success metrics: accuracy, precision, recall, F-measure, number of learnable parameters, training, and prediction time. Eventually, the experimental results reveal that DiatomNet outperforms the other models regarding all metrics with just a few exceptions. Therefore, it is a lightweight but strong candidate for diatom classification tasks.
Collapse
Affiliation(s)
- Huseyin Gunduz
- Department of Computer Engineering, Eskisehir Technical University, Eskisehir, Turkiye
| | - Serkan Gunal
- Department of Computer Engineering, Eskisehir Technical University, Eskisehir, Turkiye
| |
Collapse
|
2
|
Tournois L, Hatsch D, Ludes B, Delabarde T. Automatic detection and identification of diatoms in complex background for suspected drowning cases through object detection models. Int J Legal Med 2024; 138:659-670. [PMID: 37804333 DOI: 10.1007/s00414-023-03096-w] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/14/2023] [Accepted: 09/14/2023] [Indexed: 10/09/2023]
Abstract
The diagnosis of drowning is one of the most difficult tasks in forensic medicine. The diatom test is a complementary analysis method that may help the forensic pathologist in the diagnosis of drowning and the localization of the drowning site. This test consists in detecting or identifying diatoms, unicellular algae, in tissue and water samples. In order to observe diatoms under light microscopy, those samples may be digested by enzymes such as proteinase K. However, this digestion method may leave high amounts of debris, leading thus to a difficult detection and identification of diatoms. To the best of our knowledge, no model is proved to detect and identify accurately diatom species observed in highly complex backgrounds under light microscopy. Therefore, a novel method of model development for diatom detection and identification in a forensic context, based on sequential transfer learning of object detection models, is proposed in this article. The best resulting models are able to detect and identify up to 50 species of forensically relevant diatoms with an average precision and an average recall ranging from 0.7 to 1 depending on the concerned species. The models were developed by sequential transfer learning and globally outperformed those developed by traditional transfer learning. The best model of diatom species identification is expected to be used in routine at the Medicolegal Institute of Paris.
Collapse
Affiliation(s)
- Laurent Tournois
- UMR 8045 BABEL, Université Paris Cité, CNRS, 75012, Paris, France.
- BioSilicium, Riom, France.
| | | | - Bertrand Ludes
- UMR 8045 BABEL, Université Paris Cité, CNRS, 75012, Paris, France
- Institut Médico-Légal de Paris, Paris, France
| | - Tania Delabarde
- UMR 8045 BABEL, Université Paris Cité, CNRS, 75012, Paris, France
- Institut Médico-Légal de Paris, Paris, France
| |
Collapse
|
3
|
Pu S, Zhang F, Shu Y, Fu W. Microscopic image recognition of diatoms based on deep learning. JOURNAL OF PHYCOLOGY 2023; 59:1166-1178. [PMID: 37994558 DOI: 10.1111/jpy.13390] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/30/2023] [Revised: 09/03/2023] [Accepted: 09/05/2023] [Indexed: 11/24/2023]
Abstract
Diatoms are a crucial component in the study of aquatic ecosystems and ancient environmental records. However, traditional methods for identifying diatoms, such as morphological taxonomy and molecular detection, are costly, are time consuming, and have limitations. To address these issues, we developed an extensive collection of diatom images, consisting of 7983 images from 160 genera and 1042 species, which we expanded to 49,843 through preprocessing, segmentation, and data augmentation. Our study compared the performance of different algorithms, including backbones, batch sizes, dynamic data augmentation, and static data augmentation on experimental results. We determined that the ResNet152 network outperformed other networks, producing the most accurate results with top-1 and top-5 accuracies of 85.97% and 95.26%, respectively, in identifying 1042 diatom species. Additionally, we propose a method that combines model prediction and cosine similarity to enhance the model's performance in low-probability predictions, achieving an 86.07% accuracy rate in diatom identification. Our research contributes significantly to the recognition and classification of diatom images and has potential applications in water quality assessment, ecological monitoring, and detecting changes in aquatic biodiversity.
Collapse
Affiliation(s)
- Siyue Pu
- College of Computer and Information Engineering (College of Artificial Intelligence), Nanjing Tech University, Nanjing, China
| | - Fan Zhang
- Ocean College, Zhejiang University, Zhoushan, China
- Kavli Institute for Astrophysics and Space Research Center, Massachusettes Institute of Technology, Cambridge, Massachusetts, USA
| | - Yuexuan Shu
- Ocean College, Zhejiang University, Zhoushan, China
| | - Weiqi Fu
- Ocean College, Zhejiang University, Zhoushan, China
- Center for Systems Biology and Faculty of Industrial Engineering, Mechanical Engineering and Computer Science, School of Engineering and Natural Sciences, University of Iceland, Reykjavik, Iceland
| |
Collapse
|
4
|
Zhang J, Vieira DN, Cheng Q, Zhu Y, Deng K, Zhang J, Qin Z, Sun Q, Zhang T, Ma K, Zhang X, Huang P. DiatomNet v1.0: A novel approach for automatic diatom testing for drowning diagnosis in forensically biomedical application. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2023; 232:107434. [PMID: 36871544 DOI: 10.1016/j.cmpb.2023.107434] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/25/2022] [Revised: 09/11/2022] [Accepted: 02/18/2023] [Indexed: 06/18/2023]
Abstract
BACKGROUND AND OBJECTIVE Diatom testing is supportive for drowning diagnosis in forensic medicine. However, it is very time-consuming and labor-intensive for technicians to identify microscopically a handful of diatoms in sample smears, especially under complex observable backgrounds. Recently, we successfully developed a software, named DiatomNet v1.0 intended to automatically identify diatom frustules in a whole slide under a clear background. Here, we introduced this new software and performed a validation study to elucidate how DiatomNet v1.0 improved its performance with the influence of visible impurities. METHODS DiatomNet v1.0 has an intuitive, user-friendly and easy-to-learn graphical user interface (GUI) built in the Drupal and its core architecture for slide analysis including a convolutional neural network (CNN) is written in Python language. The build-in CNN model was evaluated for diatom identification under very complex observable backgrounds with mixtures of common impurities, including carbon pigments and sand sediments. Compared to the original model, the enhanced model following optimization with limited new datasets was evaluated systematically by independent testing and random control trials (RCTs). RESULTS In independent testing, the original DiatomNet v1.0 was moderately affected, especially when higher densities of impurities existed, and achieved a low recall of 0.817 and F1 score of 0.858 but good precision of 0.905. Following transfer learning with limited new datasets, the enhanced version had better results, with recall and F1 score values of 0.968. A comparative study on real slides showed that the upgraded DiatomNet v1.0 obtained F1 scores of 0.86 and 0.84 for carbon pigment and sand sediment, respectively, slightly worse than manual identification (carbon pigment: 0.91; sand sediment: 0.86), but much less time was needed. CONCLUSIONS The study verified that forensic diatom testing with aid of DiatomNet v1.0 is much more efficient than traditionally manual identification even under complex observable backgrounds. In terms of forensic diatom testing, we proposed a suggested standard on build-in model optimization and evaluation to strengthen the software's generalization in potentially complex conditions.
Collapse
Affiliation(s)
- Ji Zhang
- Shanghai Key Laboratory of Forensic Medicine, Shanghai Forensic Service Platform, Academy of Forensic Science, Ministry of Justice, Shanghai, P.R. China
| | - Duarte Nuno Vieira
- Department of Forensic Medicine, Ethics and Medical Law, Faculty of Medicine, University of Coimbra, Coimbra, Portugal
| | - Qi Cheng
- Shanghai Key Laboratory of Forensic Medicine, Shanghai Forensic Service Platform, Academy of Forensic Science, Ministry of Justice, Shanghai, P.R. China
| | - Yongzheng Zhu
- Shanghai Key Laboratory of Forensic Medicine, Shanghai Forensic Service Platform, Academy of Forensic Science, Ministry of Justice, Shanghai, P.R. China
| | - Kaifei Deng
- Shanghai Key Laboratory of Forensic Medicine, Shanghai Forensic Service Platform, Academy of Forensic Science, Ministry of Justice, Shanghai, P.R. China
| | - Jianhua Zhang
- Shanghai Key Laboratory of Forensic Medicine, Shanghai Forensic Service Platform, Academy of Forensic Science, Ministry of Justice, Shanghai, P.R. China
| | - Zhiqiang Qin
- Shanghai Key Laboratory of Forensic Medicine, Shanghai Forensic Service Platform, Academy of Forensic Science, Ministry of Justice, Shanghai, P.R. China
| | - Qiran Sun
- Shanghai Key Laboratory of Forensic Medicine, Shanghai Forensic Service Platform, Academy of Forensic Science, Ministry of Justice, Shanghai, P.R. China
| | - Tianye Zhang
- Shanghai Key Laboratory of Crime Scene Evidence, Institute of Criminal Science and Technology, Shanghai Municipal Public Security Bureau, Shanghai, P.R. China
| | - Kaijun Ma
- Shanghai Key Laboratory of Crime Scene Evidence, Institute of Criminal Science and Technology, Shanghai Municipal Public Security Bureau, Shanghai, P.R. China.
| | - Xiaofeng Zhang
- School of Medicine, Shanghai University, Shanghai, P.R. China.
| | - Ping Huang
- Shanghai Key Laboratory of Forensic Medicine, Shanghai Forensic Service Platform, Academy of Forensic Science, Ministry of Justice, Shanghai, P.R. China.
| |
Collapse
|
5
|
Kloster M, Burfeid-Castellanos AM, Langenkämper D, Nattkemper TW, Beszteri B. Improving deep learning-based segmentation of diatoms in gigapixel-sized virtual slides by object-based tile positioning and object integrity constraint. PLoS One 2023; 18:e0272103. [PMID: 36827378 PMCID: PMC9956069 DOI: 10.1371/journal.pone.0272103] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/11/2022] [Accepted: 10/22/2022] [Indexed: 02/26/2023] Open
Abstract
Diatoms represent one of the morphologically and taxonomically most diverse groups of microscopic eukaryotes. Light microscopy-based taxonomic identification and enumeration of frustules, the silica shells of these microalgae, is broadly used in aquatic ecology and biomonitoring. One key step in emerging digital variants of such investigations is segmentation, a task that has been addressed before, but usually in manually captured megapixel-sized images of individual diatom cells with a mostly clean background. In this paper, we applied deep learning-based segmentation methods to gigapixel-sized, high-resolution scans of diatom slides with a realistically cluttered background. This setup requires large slide scans to be subdivided into small images (tiles) to apply a segmentation model to them. This subdivision (tiling), when done using a sliding window approach, often leads to cropping relevant objects at the boundaries of individual tiles. We hypothesized that in the case of diatom analysis, reducing the amount of such cropped objects in the training data can improve segmentation performance by allowing for a better discrimination of relevant, intact frustules or valves from small diatom fragments, which are considered irrelevant when counting diatoms. We tested this hypothesis by comparing a standard sliding window / fixed-stride tiling approach with two new approaches we term object-based tile positioning with and without object integrity constraint. With all three tiling approaches, we trained Mask-R-CNN and U-Net models with different amounts of training data and compared their performance. Object-based tiling with object integrity constraint led to an improvement in pixel-based precision by 12-17 percentage points without substantially impairing recall when compared with standard sliding window tiling. We thus propose that training segmentation models with object-based tiling schemes can improve diatom segmentation from large gigapixel-sized images but could potentially also be relevant for other image domains.
Collapse
Affiliation(s)
- Michael Kloster
- Department of Phycology, Faculty of Biology, University of Duisburg-Essen, Essen, Germany
- * E-mail:
| | | | - Daniel Langenkämper
- Biodata Mining Group, Faculty of Technology, Bielefeld University, Bielefeld, Germany
| | - Tim W. Nattkemper
- Biodata Mining Group, Faculty of Technology, Bielefeld University, Bielefeld, Germany
| | - Bánk Beszteri
- Department of Phycology, Faculty of Biology, University of Duisburg-Essen, Essen, Germany
| |
Collapse
|
6
|
Refined Automatic Brain Tumor Classification Using Hybrid Convolutional Neural Networks for MRI Scans. Diagnostics (Basel) 2023; 13:diagnostics13050864. [PMID: 36900008 PMCID: PMC10001035 DOI: 10.3390/diagnostics13050864] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/18/2023] [Revised: 02/19/2023] [Accepted: 02/20/2023] [Indexed: 03/03/2023] Open
Abstract
Refined hybrid convolutional neural networks are proposed in this work for classifying brain tumor classes based on MRI scans. A dataset of 2880 T1-weighted contrast-enhanced MRI brain scans are used. The dataset contains three main classes of brain tumors: gliomas, meningiomas, and pituitary tumors, as well as a class of no tumors. Firstly, two pre-trained, fine-tuned convolutional neural networks, GoogleNet and AlexNet, were used for classification process, with validation and classification accuracy being 91.5% and 90.21%, respectively. Then, to improving the performance of the fine-tuning AlexNet, two hybrid networks (AlexNet-SVM and AlexNet-KNN) were applied. These hybrid networks achieved 96.9% and 98.6% validation and accuracy, respectively. Thus, the hybrid network AlexNet-KNN was shown to be able to apply the classification process of the present data with high accuracy. After exporting these networks, a selected dataset was employed for testing process, yielding accuracies of 88%, 85%, 95%, and 97% for the fine-tuned GoogleNet, the fine-tuned AlexNet, AlexNet-SVM, and AlexNet-KNN, respectively. The proposed system would help for automatic detection and classification of the brain tumor from the MRI scans and safe the time for the clinical diagnosis.
Collapse
|
7
|
Gong X, Ma C, Sun B, Zhang J. An Efficient Self-Organized Detection System for Algae. SENSORS (BASEL, SWITZERLAND) 2023; 23:1609. [PMID: 36772648 PMCID: PMC9920197 DOI: 10.3390/s23031609] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 12/27/2022] [Revised: 01/25/2023] [Accepted: 01/31/2023] [Indexed: 06/18/2023]
Abstract
Algal blooms have seriously affected the production and life of people and real-time detection of algae in water samples is a powerful measure to prevent algal blooms. The traditional manual detection of algae with a microscope is extremely time-consuming. In recent years, although there have been many studies using deep learning to classify and detect algae, most of them have focused on the relatively simple task of algal classification. In addition, some existing algal detection studies not only use small datasets containing limited algal species, but also only prove that object detection algorithms can be applied to algal detection tasks. These studies cannot implement the real-time detection of algae and timely warning of algal blooms. Therefore, this paper proposes an efficient self-organized detection system for algae. Benefiting from this system, we propose an interactive method to generate the algal detection dataset containing 28,329 images, 562,512 bounding boxes and 54 genera. Then, based on this dataset, we not only explore and compare the performance of 10 different versions of state-of-the-art object detection algorithms for algal detection, but also tune the detection system we built to its optimum state. In practical application, the system not only has good algal detection results, but also can complete the scanning, photographing and detection of a 2 cm × 2 cm, 0.1 mL algal slide specimen within five minutes (the resolution is 0.25886 μm/pixel); such a task requires a well-trained algal expert to work continuously for more than three hours. The efficient algal self-organized detection system we built makes it possible to detect algae in real time. In the future, with the help of IoT, we can use various smart sensors, actuators and intelligent controllers to achieve real-time collection and wireless transmission of algal data, use the efficient algal self-organized detection system we built to implement real-time algal detection and upload the detection results to the cloud to realize timely warning of algal blooms.
Collapse
Affiliation(s)
- Xingrui Gong
- School of Computer Science and Engineering, Faculty of Innovation Engineering, Macau University of Science and Technology, Macau 999078, China
| | - Chao Ma
- Department of Engineering Science, Faculty of Innovation Engineering, Macau University of Science and Technology, Macau 999078, China
| | - Beili Sun
- Jiangsu Metabio Science & Technology Co., Ltd., Wuxi 214028, China
- Wuxi Key Laboratory of Biochips, Southeast University Wuxi Branch, Wuxi 214135, China
| | - Junyi Zhang
- Jiangsu Wuxi Environmental Monitoring Center, Wuxi 214121, China
- School of Environmental and Civil Engineering, Jiangnan University, Wuxi 214122, China
| |
Collapse
|
8
|
Applications of artificial neural networks in microorganism image analysis: a comprehensive review from conventional multilayer perceptron to popular convolutional neural network and potential visual transformer. Artif Intell Rev 2023; 56:1013-1070. [PMID: 35528112 PMCID: PMC9066147 DOI: 10.1007/s10462-022-10192-7] [Citation(s) in RCA: 32] [Impact Index Per Article: 32.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/02/2023]
Abstract
Microorganisms are widely distributed in the human daily living environment. They play an essential role in environmental pollution control, disease prevention and treatment, and food and drug production. The analysis of microorganisms is essential for making full use of different microorganisms. The conventional analysis methods are laborious and time-consuming. Therefore, the automatic image analysis based on artificial neural networks is introduced to optimize it. However, the automatic microorganism image analysis faces many challenges, such as the requirement of a robust algorithm caused by various application occasions, insignificant features and easy under-segmentation caused by the image characteristic, and various analysis tasks. Therefore, we conduct this review to comprehensively discuss the characteristics of microorganism image analysis based on artificial neural networks. In this review, the background and motivation are introduced first. Then, the development of artificial neural networks and representative networks are presented. After that, the papers related to microorganism image analysis based on classical and deep neural networks are reviewed from the perspectives of different tasks. In the end, the methodology analysis and potential direction are discussed.
Collapse
|
9
|
Abdelaziz HA, Sallah M, Elgarayhi A, Al-Tahhan FE. Accurate automatic classification system for 3D CT images of some vertebrate remains from Egypt. JOURNAL OF TAIBAH UNIVERSITY FOR SCIENCE 2022. [DOI: 10.1080/16583655.2022.2096391] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 10/17/2022]
Affiliation(s)
- Hussien A. Abdelaziz
- Applied Mathematical Physics Research Group, Physics Department, Faculty of Science, Mansoura University, Mansoura, Egypt
| | - Mohammed Sallah
- Applied Mathematical Physics Research Group, Physics Department, Faculty of Science, Mansoura University, Mansoura, Egypt
- Higher Institute of Engineering and Technology, New Damietta, Egypt
| | - Ahmed Elgarayhi
- Applied Mathematical Physics Research Group, Physics Department, Faculty of Science, Mansoura University, Mansoura, Egypt
| | - Fatma E. Al-Tahhan
- Mathematics Department, Faculty of Science, Mansoura University, Mansoura, Egypt
| |
Collapse
|
10
|
Luo J, Ser W, Liu A, Yap P, Liedberg B, Rayatpisheh S. Low complexity and accurate Machine learning model for waterborne pathogen classification using only three handcrafted features from optofluidic images. Biomed Signal Process Control 2022. [DOI: 10.1016/j.bspc.2022.103821] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/02/2022]
|
11
|
Gu G, Gan S, Deng J, Du Y, Qiu Z, Liu J, Liu C, Zhao J. Automated diatom detection in forensic drowning diagnosis using a single shot multibox detector with plump receptive field. Appl Soft Comput 2022. [DOI: 10.1016/j.asoc.2022.108885] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/02/2022]
|
12
|
Vallez N, Bueno G, Deniz O, Blanco S. Diffeomorphic transforms for data augmentation of highly variable shape and texture objects. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2022; 219:106775. [PMID: 35397412 DOI: 10.1016/j.cmpb.2022.106775] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 09/27/2021] [Revised: 02/28/2022] [Accepted: 03/23/2022] [Indexed: 06/14/2023]
Abstract
BACKGROUND AND OBJECTIVE Training a deep convolutional neural network (CNN) for automatic image classification requires a large database with images of labeled samples. However, in some applications such as biology and medicine only a few experts can correctly categorize each sample. Experts are able to identify small changes in shape and texture which go unnoticed by untrained people, as well as distinguish between objects in the same class that present drastically different shapes and textures. This means that currently available databases are too small and not suitable to train deep learning models from scratch. To deal with this problem, data augmentation techniques are commonly used to increase the dataset size. However, typical data augmentation methods introduce artifacts or apply distortions to the original image, which instead of creating new realistic samples, obtain basic spatial variations of the original ones. METHODS We propose a novel data augmentation procedure which generates new realistic samples, by combining two samples that belong to the same class. Although the idea behind the method described in this paper is to mimic the variations that diatoms experience in different stages of their life cycle, it has also been demonstrated in glomeruli and pollen identification problems. This new data augmentation procedure is based on morphing and image registration methods that perform diffeomorphic transformations. RESULTS The proposed technique achieves an increase in accuracy over existing techniques of 0.47%, 1.47%, and 0.23% for diatom, glomeruli and pollen problems respectively. CONCLUSIONS For the Diatom dataset, the method is able to simulate the shape changes in different diatom life cycle stages, and thus, images generated resemble newly acquired samples with intermediate shapes. In fact, the other methods compared obtained worse results than those which were not using data augmentation. For the Glomeruli dataset, the method is able to add new samples with different shapes and degrees of sclerosis (through different textures). This is the case where our proposed DA method is more beneficial, when objects highly differ in both shape and texture. Finally, for the Pollen dataset, since there are only small variations between samples in a few classes and this dataset has other features such as noise which are likely to benefit other existing DA techniques, the method still shows an improvement of the results.
Collapse
Affiliation(s)
- Noelia Vallez
- VISILAB, University of Castilla-La Mancha, E.T.S. Ingenieria Industrial, Avda. Camilo Jose Cela s/n, Ciudad Real 13071, Spain.
| | - Gloria Bueno
- VISILAB, University of Castilla-La Mancha, E.T.S. Ingenieria Industrial, Avda. Camilo Jose Cela s/n, Ciudad Real 13071, Spain
| | - Oscar Deniz
- VISILAB, University of Castilla-La Mancha, E.T.S. Ingenieria Industrial, Avda. Camilo Jose Cela s/n, Ciudad Real 13071, Spain
| | - Saul Blanco
- Institute of the Environment, University of Leon, Leon E-24071, Spain
| |
Collapse
|
13
|
Species-Level Microfossil Prediction for Globotruncana genus Using Machine Learning Models. ARABIAN JOURNAL FOR SCIENCE AND ENGINEERING 2022. [DOI: 10.1007/s13369-022-06822-5] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 01/22/2023]
|
14
|
Multi-exposure microscopic image fusion-based detail enhancement algorithm. Ultramicroscopy 2022; 236:113499. [DOI: 10.1016/j.ultramic.2022.113499] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/24/2021] [Revised: 12/16/2021] [Accepted: 02/16/2022] [Indexed: 02/04/2023]
|
15
|
Microorganism image classification with circle-based Multi-Region Binarization and mutual-information-based feature selection. BIOMEDICAL ENGINEERING ADVANCES 2021. [DOI: 10.1016/j.bea.2021.100020] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/21/2022] Open
|
16
|
Central Courtyard Feature Extraction in Remote Sensing Aerial Images Using Deep Learning: A Case-Study of Iran. REMOTE SENSING 2021. [DOI: 10.3390/rs13234843] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 12/26/2022]
Abstract
Central courtyards are primary components of vernacular architecture in Iran. The directions, dimensions, ratios, and other characteristics of central courtyards are critical for studying historical passive cooling and heating solutions. Several studies on central courtyards have compared their features in different cities and climatic zones in Iran. In this study, deep learning methods for object detection and image segmentation are applied to aerial images, to extract the features of central courtyards. The case study explores aerial images of nine historical cities in Bsk, Bsh, Bwk, and Bwh Köppen climate zones. Furthermore, these features were gathered in an extensive dataset, with 26,437 samples and 76 geometric and climatic features. Additionally, the data analysis methods reveal significant correlations between various features, such as the length and width of courtyards. In all cities, the correlation coefficient between these two characteristics is approximately +0.88. Numerous mathematical equations are generated for each city and climate zone by fitting the linear regression model to these data in different cities and climate zones. These equations can be used as proposed design models to assist designers and researchers in predicting and locating the best courtyard houses in Iran’s historical regions.
Collapse
|
17
|
Deng J, Guo W, Zhao Y, Liu J, Lai R, Gu G, Zhang Y, Li Q, Liu C, Zhao J. Identification of diatom taxonomy by a combination of region-based full convolutional network, online hard example mining, and shape priors of diatoms. Int J Legal Med 2021; 135:2519-2530. [PMID: 34282483 DOI: 10.1007/s00414-021-02664-2] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/24/2021] [Accepted: 07/05/2021] [Indexed: 12/01/2022]
Abstract
Diatom test is one of the commonly used diagnostic methods for drowning in forensic pathology, which provides supportive evidence for drowning. However, in forensic practice, it is time-consuming and laborious for forensic experts to classify and count diatoms, whereas artificial intelligence (AI) is superior to human experts in processing data and carrying out classification tasks. Some AI techniques have focused on searching diatoms and classifying diatoms. But, they either could not classify diatoms correctly or were time-consuming. Conventional detection deep network has been used to overcome these problems but failed to detect the occluded diatoms and the diatoms similar to the background heavily, which could lead to false positives or false negatives. In order to figure out the problems above, an improved region-based full convolutional network (R-FCN) with online hard example mining and the shape prior of diatoms was proposed. The online hard example mining (OHEM) was coupled with the R-FCN to boost the capacity of detecting the occluded diatoms and the diatoms similar to the background heavily and the priors of the shape of the common diatoms were explored and introduced to the anchor generation strategy of the region proposal network in the R-FCN to locate the diatoms precisely. The results showed that the proposed approach significantly outperforms several state-of-the-art methods and could detect the diatom precisely without missing the occluded diatoms and the diatoms similar to the background heavily. From the study, we could conclude that (1) the proposed model can locate the position and identify the genera of common diatoms more accurately; (2) this method can reduce the false positives or false negatives in forensic practice; and (3) it is a time-saving method and can be introduced.
Collapse
Affiliation(s)
- Jiehang Deng
- School of Computers, Guangdong University of Technology, Guangzhou, 510006, China
| | - Wenquan Guo
- School of Computers, Guangdong University of Technology, Guangzhou, 510006, China
| | - Youwei Zhao
- School of Computers, Guangdong University of Technology, Guangzhou, 510006, China
| | - Jingjian Liu
- Kunming Medical University, Chunrong Road West 1168, Chenggong District, Kunming, China
| | - Runhao Lai
- School of Computers, Guangdong University of Technology, Guangzhou, 510006, China
| | - Guosheng Gu
- School of Computers, Guangdong University of Technology, Guangzhou, 510006, China.
| | - Yalong Zhang
- College of Electrical and Information Engineering, Quzhou University, Quzhou, 324000, China
| | - Qi Li
- Faculty of Forensic Medicine, Zhongshan School of Medicine, Sun Yat-Sen University & Guangdong Province Translational Forensic Medicine Engineering Technology Research Center, Zhongshan 2nd Road 74, Yuexiu District, Guangzhou, People's Republic of China
| | - Chao Liu
- Faculty of Forensic Medicine, Zhongshan School of Medicine, Sun Yat-Sen University & Guangdong Province Translational Forensic Medicine Engineering Technology Research Center, Zhongshan 2nd Road 74, Yuexiu District, Guangzhou, People's Republic of China. .,Guangzhou Forensic Science Institute & Key Laboratory of Forensic Pathology, Ministry of Public Security, Baiyun Avenue 1708, Baiyun District, Guangzhou, People's Republic of China.
| | - Jian Zhao
- Faculty of Forensic Medicine, Zhongshan School of Medicine, Sun Yat-Sen University & Guangdong Province Translational Forensic Medicine Engineering Technology Research Center, Zhongshan 2nd Road 74, Yuexiu District, Guangzhou, People's Republic of China. .,Guangzhou Forensic Science Institute & Key Laboratory of Forensic Pathology, Ministry of Public Security, Baiyun Avenue 1708, Baiyun District, Guangzhou, People's Republic of China.
| |
Collapse
|
18
|
Dhindsa A, Bhatia S, Agrawal S, Sohi BS. An Improvised Machine Learning Model Based on Mutual Information Feature Selection Approach for Microbes Classification. ENTROPY (BASEL, SWITZERLAND) 2021; 23:257. [PMID: 33672252 PMCID: PMC7927045 DOI: 10.3390/e23020257] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 01/14/2021] [Revised: 02/10/2021] [Accepted: 02/20/2021] [Indexed: 12/11/2022]
Abstract
The accurate classification of microbes is critical in today's context for monitoring the ecological balance of a habitat. Hence, in this research work, a novel method to automate the process of identifying microorganisms has been implemented. To extract the bodies of microorganisms accurately, a generalized segmentation mechanism which consists of a combination of convolution filter (Kirsch) and a variance-based pixel clustering algorithm (Otsu) is proposed. With exhaustive corroboration, a set of twenty-five features were identified to map the characteristics and morphology for all kinds of microbes. Multiple techniques for feature selection were tested and it was found that mutual information (MI)-based models gave the best performance. Exhaustive hyperparameter tuning of multilayer layer perceptron (MLP), k-nearest neighbors (KNN), quadratic discriminant analysis (QDA), logistic regression (LR), and support vector machine (SVM) was done. It was found that SVM radial required further improvisation to attain a maximum possible level of accuracy. Comparative analysis between SVM and improvised SVM (ISVM) through a 10-fold cross validation method ultimately showed that ISVM resulted in a 2% higher performance in terms of accuracy (98.2%), precision (98.2%), recall (98.1%), and F1 score (98.1%).
Collapse
Affiliation(s)
- Anaahat Dhindsa
- Department of Electronics and Communication Engineering, Chandigarh University, Gharuan, Punjab 140413, India;
- University Institute of Engineering and Technology, Panjab University, Chandigarh 160014, India;
| | - Sanjay Bhatia
- Post Graduate Department of Zoology, University of Jammu, Kashmir 180006, India;
| | - Sunil Agrawal
- University Institute of Engineering and Technology, Panjab University, Chandigarh 160014, India;
| | - Balwinder Singh Sohi
- Department of Electronics and Communication Engineering, Chandigarh University, Gharuan, Punjab 140413, India;
| |
Collapse
|
19
|
Zhang J, Zhou Y, Vieira DN, Cao Y, Deng K, Cheng Q, Zhu Y, Zhang J, Qin Z, Ma K, Chen Y, Huang P. An efficient method for building a database of diatom populations for drowning site inference using a deep learning algorithm. Int J Legal Med 2021; 135:817-827. [PMID: 33392655 DOI: 10.1007/s00414-020-02497-5] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/03/2020] [Accepted: 12/17/2020] [Indexed: 10/22/2022]
Abstract
Seasonal or monthly databases of the diatom populations in specific bodies of water are needed to infer the drowning site of a drowned body. However, existing diatom testing methods are laborious, time-consuming, and costly and usually require specific expertise. In this study, we developed an artificial intelligence (AI)-based system as a substitute for manual morphological examination capable of identifying and classifying diatoms at the species level. Within two days, the system collected information on diatom profiles in the Huangpu and Suzhou Rivers of Shanghai, China. In an animal experiment, the similarities of diatom profiles between lung tissues and water samples were evaluated through a modified Jensen-Shannon (JS) divergence measure for drowning site inference, reaching a prediction accuracy of 92.31%. Considering its high efficiency and simplicity, our proposed method is believed to be more applicable than existing methods for seasonal or monthly water monitoring of diatom populations from sections of interconnected rivers, which would help police narrow the investigation scope to confirm the identity of an immersed body.
Collapse
Affiliation(s)
- Ji Zhang
- Shanghai Key Laboratory of Forensic Medicine, Shanghai Forensic Service Platform, Academy of Forensic Science, Ministry of Justice, Shanghai, People's Republic of China
| | - Yuanyuan Zhou
- Shanghai Key Laboratory of Forensic Medicine, Shanghai Forensic Service Platform, Academy of Forensic Science, Ministry of Justice, Shanghai, People's Republic of China.,Department of Forensic Medicine, Inner Mongolia Medical University, Huhhot, Inner Mongolia, People's Republic of China
| | - Duarte Nuno Vieira
- Department of Forensic Medicine, Ethics and Medical Law, Faculty of Medicine, University of Coimbra, Coimbra, Portugal
| | - Yongjie Cao
- Department of Forensic Medicine, Nanjing Medical University, Nanjing, Jiangsu, People's Republic of China
| | - Kaifei Deng
- Shanghai Key Laboratory of Forensic Medicine, Shanghai Forensic Service Platform, Academy of Forensic Science, Ministry of Justice, Shanghai, People's Republic of China
| | - Qi Cheng
- Department of Forensic Medicine, Guizhou Medical University, Guiyang, Guizhou, People's Republic of China
| | - Yongzheng Zhu
- School of Forensic Medicine, Shanxi Medical University, Taiyuan, Shanxi, People's Republic of China
| | - Jianhua Zhang
- Shanghai Key Laboratory of Forensic Medicine, Shanghai Forensic Service Platform, Academy of Forensic Science, Ministry of Justice, Shanghai, People's Republic of China
| | - Zhiqiang Qin
- Shanghai Key Laboratory of Forensic Medicine, Shanghai Forensic Service Platform, Academy of Forensic Science, Ministry of Justice, Shanghai, People's Republic of China
| | - Kaijun Ma
- Shanghai Key Laboratory of Crime Scene Evidence, Institute of Criminal Science and Technology, Shanghai Municipal Public Security Bureau, Shanghai, People's Republic of China.
| | - Yijiu Chen
- Shanghai Key Laboratory of Forensic Medicine, Shanghai Forensic Service Platform, Academy of Forensic Science, Ministry of Justice, Shanghai, People's Republic of China.
| | - Ping Huang
- Shanghai Key Laboratory of Forensic Medicine, Shanghai Forensic Service Platform, Academy of Forensic Science, Ministry of Justice, Shanghai, People's Republic of China.
| |
Collapse
|
20
|
Automatic Detection of Freshwater Phytoplankton Specimens in Conventional Microscopy Images. SENSORS 2020; 20:s20226704. [PMID: 33238566 PMCID: PMC7700267 DOI: 10.3390/s20226704] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 10/15/2020] [Revised: 11/11/2020] [Accepted: 11/20/2020] [Indexed: 01/24/2023]
Abstract
Water safety and quality can be compromised by the proliferation of toxin-producing phytoplankton species, requiring continuous monitoring of water sources. This analysis involves the identification and counting of these species which requires broad experience and knowledge. The automatization of these tasks is highly desirable as it would release the experts from tedious work, eliminate subjective factors, and improve repeatability. Thus, in this preliminary work, we propose to advance towards an automatic methodology for phytoplankton analysis in digital images of water samples acquired using regular microscopes. In particular, we propose a novel and fully automatic method to detect and segment the existent phytoplankton specimens in these images using classical computer vision algorithms. The proposed method is able to correctly detect sparse colonies as single phytoplankton candidates, thanks to a novel fusion algorithm, and is able to differentiate phytoplankton specimens from other image objects in the microscope samples (like minerals, bubbles or detritus) using a machine learning based approach that exploits texture and colour features. Our preliminary experiments demonstrate that the proposed method provides satisfactory and accurate results.
Collapse
|
21
|
Memmolo P, Carcagnì P, Bianco V, Merola F, Goncalves da Silva Junior A, Garcia Goncalves LM, Ferraro P, Distante C. Learning Diatoms Classification from a Dry Test Slide by Holographic Microscopy. SENSORS 2020; 20:s20216353. [PMID: 33171757 PMCID: PMC7664373 DOI: 10.3390/s20216353] [Citation(s) in RCA: 9] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 10/14/2020] [Revised: 11/03/2020] [Accepted: 11/05/2020] [Indexed: 01/05/2023]
Abstract
Diatoms are among the dominant phytoplankters in marine and freshwater habitats, and important biomarkers of water quality, making their identification and classification one of the current challenges for environmental monitoring. To date, taxonomy of the species populating a water column is still conducted by marine biologists on the basis of their own experience. On the other hand, deep learning is recognized as the elective technique for solving image classification problems. However, a large amount of training data is usually needed, thus requiring the synthetic enlargement of the dataset through data augmentation. In the case of microalgae, the large variety of species that populate the marine environments makes it arduous to perform an exhaustive training that considers all the possible classes. However, commercial test slides containing one diatom element per class fixed in between two glasses are available on the market. These are usually prepared by expert diatomists for taxonomy purposes, thus constituting libraries of the populations that can be found in oceans. Here we show that such test slides are very useful for training accurate deep Convolutional Neural Networks (CNNs). We demonstrate the successful classification of diatoms based on a proper CNNs ensemble and a fully augmented dataset, i.e., creation starting from one single image per class available from a commercial glass slide containing 50 fixed species in a dry setting. This approach avoids the time-consuming steps of water sampling and labeling by skilled marine biologists. To accomplish this goal, we exploit the holographic imaging modality, which permits the accessing of a quantitative phase-contrast maps and a posteriori flexible refocusing due to its intrinsic 3D imaging capability. The network model is then validated by using holographic recordings of live diatoms imaged in water samples i.e., in their natural wet environmental condition.
Collapse
Affiliation(s)
- Pasquale Memmolo
- Institute of Applied Sciences and Intelligent Systems (ISASI) National Research Council (CNR) of Italy, Via Campi Flegrei 34, 80078 Pozzuoli, NA, Italy; (P.M.); (F.M.); (P.F.)
| | - Pierluigi Carcagnì
- Institute of Applied Sciences and Intelligent Systems (ISASI) National Research Council (CNR) of Italy, Via Monteorni snc University Campus, 73100 Lecce, Italy; (P.C.); (C.D.)
| | - Vittorio Bianco
- Institute of Applied Sciences and Intelligent Systems (ISASI) National Research Council (CNR) of Italy, Via Campi Flegrei 34, 80078 Pozzuoli, NA, Italy; (P.M.); (F.M.); (P.F.)
- Correspondence: ; Tel.: +39-0818675201
| | - Francesco Merola
- Institute of Applied Sciences and Intelligent Systems (ISASI) National Research Council (CNR) of Italy, Via Campi Flegrei 34, 80078 Pozzuoli, NA, Italy; (P.M.); (F.M.); (P.F.)
| | | | - Luis Marcos Garcia Goncalves
- Department of Computer Engineering and Automation, Federal University of Rio Grande do Norte, 59078 Natal, Brazil; (A.G.d.S.J.); (L.M.G.G.)
| | - Pietro Ferraro
- Institute of Applied Sciences and Intelligent Systems (ISASI) National Research Council (CNR) of Italy, Via Campi Flegrei 34, 80078 Pozzuoli, NA, Italy; (P.M.); (F.M.); (P.F.)
| | - Cosimo Distante
- Institute of Applied Sciences and Intelligent Systems (ISASI) National Research Council (CNR) of Italy, Via Monteorni snc University Campus, 73100 Lecce, Italy; (P.C.); (C.D.)
| |
Collapse
|
22
|
Jiang L, Xiao C, Zhao J, Jiang T, Lin J, Xu Q, Liu C, Cai W. Development of 18S rRNA gene arrays for forensic detection of diatoms. Forensic Sci Int 2020; 317:110482. [PMID: 33142211 DOI: 10.1016/j.forsciint.2020.110482] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/28/2020] [Revised: 07/27/2020] [Accepted: 08/30/2020] [Indexed: 02/07/2023]
Abstract
Diatom test is the most commonly used method to diagnose drowning in forensic laboratories. However, microscopic examination and identification of diatom frustules is time-consuming and requires taxonomic expertise. At present, the identification of drowning is still a challenge in forensic casework. In this study, we developed a novel diatom microarray based on the detection of specific 18S rRNA gene fragments of diatom species. The array covers 169 diatom species which were documented as commonly found in a wide range of fresh waters in China. Diatom arrays were prepared from species specific oligonucleotide probes targeting to variable regions of the 18S rRNA gene. We also developed an auxiliary sample preparation method for isolation of diatom DNA from tissues, which enabled detection of diatom species in real forensic samples as well as environmental waters. We applied the diatom arrays to analyze six drowned cases and eight environmental samples. The diatom arrays showed much better sensitivity and more consistent results than those of the conventional SEM methods. We discovered major discrepancies between results generated by the diatom arrays and the routinely used SEM based diatom tests. We verified the results of our diatom arrays by species specific PCR and Sanger sequencing and found that the currently used SEM diatom test method has a serious deficiency in sensitivity due to high loss rate of frustules in the sample preparation procedure. We anticipate that the application of diatom arrays will transform current forensic practice of diagnosing drowning deaths.
Collapse
Affiliation(s)
- Lin Jiang
- College of Biological Sciences and Technology, Fuzhou University, Fuzhou, 350108, China
| | - Cheng Xiao
- School of Forensic Medicine, Southern Medical University, Guangzhou, 510515, China; Guangzhou Forensic Science Institute, Guangdong Province Key Laboratory of Forensic Genetics, Guangzhou, 510030, China
| | - Jian Zhao
- Guangzhou Forensic Science Institute, Guangdong Province Key Laboratory of Forensic Genetics, Guangzhou, 510030, China
| | - Tao Jiang
- College of Life Science and Technology, Huazhong University of Science and Technology, Wuhan, 1037, China
| | - Jun Lin
- College of Biological Sciences and Technology, Fuzhou University, Fuzhou, 350108, China
| | - Quyi Xu
- Guangzhou Forensic Science Institute, Guangdong Province Key Laboratory of Forensic Genetics, Guangzhou, 510030, China
| | - Chao Liu
- School of Forensic Medicine, Southern Medical University, Guangzhou, 510515, China; Guangzhou Forensic Science Institute, Guangdong Province Key Laboratory of Forensic Genetics, Guangzhou, 510030, China
| | - Weiwen Cai
- College of Biological Sciences and Technology, Fuzhou University, Fuzhou, 350108, China.
| |
Collapse
|
23
|
Kloster M, Langenkämper D, Zurowietz M, Beszteri B, Nattkemper TW. Deep learning-based diatom taxonomy on virtual slides. Sci Rep 2020; 10:14416. [PMID: 32879374 PMCID: PMC7468105 DOI: 10.1038/s41598-020-71165-w] [Citation(s) in RCA: 13] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/19/2020] [Accepted: 07/15/2020] [Indexed: 11/18/2022] Open
Abstract
Deep convolutional neural networks are emerging as the state of the art method for supervised classification of images also in the context of taxonomic identification. Different morphologies and imaging technologies applied across organismal groups lead to highly specific image domains, which need customization of deep learning solutions. Here we provide an example using deep convolutional neural networks (CNNs) for taxonomic identification of the morphologically diverse microalgal group of diatoms. Using a combination of high-resolution slide scanning microscopy, web-based collaborative image annotation and diatom-tailored image analysis, we assembled a diatom image database from two Southern Ocean expeditions. We use these data to investigate the effect of CNN architecture, background masking, data set size and possible concept drift upon image classification performance. Surprisingly, VGG16, a relatively old network architecture, showed the best performance and generalizing ability on our images. Different from a previous study, we found that background masking slightly improved performance. In general, training only a classifier on top of convolutional layers pre-trained on extensive, but not domain-specific image data showed surprisingly high performance (F1 scores around 97%) with already relatively few (100–300) examples per class, indicating that domain adaptation to a novel taxonomic group can be feasible with a limited investment of effort.
Collapse
Affiliation(s)
- Michael Kloster
- Department of Phycology, Faculty of Biology, University of Duisburg-Essen, Essen, Germany. .,Biodata Mining Group, Faculty of Technology, Bielefeld University, Bielefeld, Germany.
| | - Daniel Langenkämper
- Biodata Mining Group, Faculty of Technology, Bielefeld University, Bielefeld, Germany
| | - Martin Zurowietz
- Biodata Mining Group, Faculty of Technology, Bielefeld University, Bielefeld, Germany
| | - Bánk Beszteri
- Department of Phycology, Faculty of Biology, University of Duisburg-Essen, Essen, Germany
| | - Tim W Nattkemper
- Biodata Mining Group, Faculty of Technology, Bielefeld University, Bielefeld, Germany
| |
Collapse
|
24
|
A Low-Cost Automated Digital Microscopy Platform for Automatic Identification of Diatoms. APPLIED SCIENCES-BASEL 2020. [DOI: 10.3390/app10176033] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/16/2022]
Abstract
Currently, microalgae (i.e., diatoms) constitute a generally accepted bioindicator of water quality and therefore provide an index of the status of biological ecosystems. Diatom detection for specimen counting and sample classification are two difficult time-consuming tasks for the few existing expert diatomists. To mitigate this challenge, in this work, we propose a fully operative low-cost automated microscope, integrating algorithms for: (1) stage and focus control, (2) image acquisition (slide scanning, stitching, contrast enhancement), and (3) diatom detection and a prospective specimen classification (among 80 taxa). Deep learning algorithms have been applied to overcome the difficult selection of image descriptors imposed by classical machine learning strategies. With respect to the mentioned strategies, the best results were obtained by deep neural networks with a maximum precision of 86% (with the YOLO network) for detection and 99.51% for classification, among 80 different species (with the AlexNet network). All the developed operational modules are integrated and controlled by the user from the developed graphical user interface running in the main controller. With the developed operative platform, it is noteworthy that this work provides a quite useful toolbox for phycologists in their daily challenging tasks to identify and classify diatoms.
Collapse
|
25
|
Krause LMK, Koc J, Rosenhahn B, Rosenhahn A. Fully Convolutional Neural Network for Detection and Counting of Diatoms on Coatings after Short-Term Field Exposure. ENVIRONMENTAL SCIENCE & TECHNOLOGY 2020; 54:10022-10030. [PMID: 32663392 DOI: 10.1021/acs.est.0c01982] [Citation(s) in RCA: 10] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/11/2023]
Abstract
While the use of deep learning is a valuable technology for automatic detection systems for medical data and images, the biofouling community is still lacking an analytical tool for the detection and counting of diatoms on samples after short-term field exposure. In this work, a fully convolutional neural network was implemented as a fast and simple approach to detect diatoms on two-channel (fluorescence and phase-contrast) microscopy images by predicting bounding boxes. The developed approach performs well with only a small number of trainable parameters and a F1 score of 0.82. Counting diatoms was evaluated on a data set of 600 microscopy images of three different surface chemistries (hydrophilic and hydrophobic) and is very similar to counting by humans while demanding only a fraction of the analysis time.
Collapse
Affiliation(s)
- Lutz M K Krause
- Analytical Chemistry-Biointerfaces, Ruhr University Bochum, 44801 Bochum, Germany
| | - Julian Koc
- Analytical Chemistry-Biointerfaces, Ruhr University Bochum, 44801 Bochum, Germany
| | - Bodo Rosenhahn
- Institute for Information Processing, Leibniz University Hannover, 30167 Hannover, Germany
| | - Axel Rosenhahn
- Analytical Chemistry-Biointerfaces, Ruhr University Bochum, 44801 Bochum, Germany
| |
Collapse
|
26
|
Zhou Y, Cao Y, Huang J, Deng K, Ma K, Zhang T, Chen L, Zhang J, Huang P. Research advances in forensic diatom testing. Forensic Sci Res 2020; 5:98-105. [PMID: 32939425 PMCID: PMC7476611 DOI: 10.1080/20961790.2020.1718901] [Citation(s) in RCA: 19] [Impact Index Per Article: 4.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/20/2019] [Revised: 10/27/2019] [Accepted: 12/11/2019] [Indexed: 10/25/2022] Open
Abstract
In forensic practice, it is difficult to determine whether a dead body in the water resulted from drowning or from disposal after death. Diatom testing is currently an important supporting technique for the determination of death by drowning and of drowning sites, even though it is a time-consuming and laborious task. This article reviews the development of diatom testing over the decades and discusses a new method for the potential application of deep learning in diatom testing.
Collapse
Affiliation(s)
- Yuanyuan Zhou
- Shanghai Key Laboratory of Forensic Medicine, Shanghai Forensic Service Platform, Academy of Forensic Science, Ministry of Justice, Shanghai, China
- Department of Forensic Medicine, Inner Mongolia Medical University, Huhhot, China
| | - Yongjie Cao
- Department of Forensic Medicine, Nanjing Medical University, Nanjing, China
| | - Jiao Huang
- Department of Forensic Medicine, Xuzhou Medical University, Xuzhou, China
| | - Kaifei Deng
- Shanghai Key Laboratory of Forensic Medicine, Shanghai Forensic Service Platform, Academy of Forensic Science, Ministry of Justice, Shanghai, China
| | - Kaijun Ma
- Institute of Forensic Science Shanghai Municipal Public Security Bureau, Shanghai, China
| | - Tianye Zhang
- Institute of Forensic Science Shanghai Municipal Public Security Bureau, Shanghai, China
| | - Liqin Chen
- Department of Forensic Medicine, Inner Mongolia Medical University, Huhhot, China
| | - Ji Zhang
- Shanghai Key Laboratory of Forensic Medicine, Shanghai Forensic Service Platform, Academy of Forensic Science, Ministry of Justice, Shanghai, China
| | - Ping Huang
- Shanghai Key Laboratory of Forensic Medicine, Shanghai Forensic Service Platform, Academy of Forensic Science, Ministry of Justice, Shanghai, China
| |
Collapse
|
27
|
Zhou Y, Zhang J, Huang J, Deng K, Zhang J, Qin Z, Wang Z, Zhang X, Tuo Y, Chen L, Chen Y, Huang P. Digital whole-slide image analysis for automated diatom test in forensic cases of drowning using a convolutional neural network algorithm. Forensic Sci Int 2019; 302:109922. [DOI: 10.1016/j.forsciint.2019.109922] [Citation(s) in RCA: 24] [Impact Index Per Article: 4.8] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/15/2019] [Revised: 08/05/2019] [Accepted: 08/06/2019] [Indexed: 01/01/2023]
|
28
|
Dunker S. Hidden Secrets Behind Dots: Improved Phytoplankton Taxonomic Resolution Using High‐Throughput Imaging Flow Cytometry. Cytometry A 2019; 95:854-868. [DOI: 10.1002/cyto.a.23870] [Citation(s) in RCA: 14] [Impact Index Per Article: 2.8] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/02/2019] [Revised: 07/16/2019] [Accepted: 07/18/2019] [Indexed: 11/12/2022]
Affiliation(s)
- Susanne Dunker
- Helmholtz‐Centre for Environmental Research – UFZ, Department Physiological Diversity, Permoserstraße 15 04318 Leipzig Germany
- German Centre for Integrative Biodiversity Research ‐ iDiv, Department Physiological Diversity, Deutscher Platz 5e 04318 Leipzig Germany
| |
Collapse
|
29
|
Sánchez C, Cristóbal G, Bueno G. Diatom identification including life cycle stages through morphological and texture descriptors. PeerJ 2019; 7:e6770. [PMID: 31086732 PMCID: PMC6487182 DOI: 10.7717/peerj.6770] [Citation(s) in RCA: 8] [Impact Index Per Article: 1.6] [Reference Citation Analysis] [Abstract] [Key Words] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/11/2018] [Accepted: 03/11/2019] [Indexed: 11/20/2022] Open
Abstract
Diatoms are unicellular algae present almost wherever there is water. Diatom identification has many applications in different fields of study, such as ecology, forensic science, etc. In environmental studies, algae can be used as a natural water quality indicator. The diatom life cycle consists of the set of stages that pass through the successive generations of each species from the initial to the senescent cells. Life cycle modeling is a complex process since in general the distribution of the parameter vectors that represent the variations that occur in this process is non-linear and of high dimensionality. In this paper, we propose to characterize the diatom life cycle by the main features that change during the algae life cycle, mainly the contour shape and the texture. Elliptical Fourier Descriptors (EFD) are used to describe the diatom contour while phase congruency and Gabor filters describe the inner ornamentation of the algae. The proposed method has been tested with a small algae dataset (eight different classes and more than 50 samples per type) using supervised and non-supervised classification techniques obtaining accuracy results up to 99% and 98% respectively.
Collapse
Affiliation(s)
- Carlos Sánchez
- Instituto de Óptica "Daza de Valdés", CSIC, Madrid, Spain
| | | | - Gloria Bueno
- VISILAB, Universidad de Castilla La Mancha, Ciudad Real, Spain
| |
Collapse
|
30
|
Deep Learning Versus Classic Methods for Multi-taxon Diatom Segmentation. PATTERN RECOGNITION AND IMAGE ANALYSIS 2019. [DOI: 10.1007/978-3-030-31332-6_30] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 01/17/2023]
|
31
|
Diatom Classification Including Morphological Adaptations Using CNNs. PATTERN RECOGNITION AND IMAGE ANALYSIS 2019. [DOI: 10.1007/978-3-030-31332-6_28] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/27/2022]
|
32
|
Dunker S, Boho D, Wäldchen J, Mäder P. Combining high-throughput imaging flow cytometry and deep learning for efficient species and life-cycle stage identification of phytoplankton. BMC Ecol 2018; 18:51. [PMID: 30509239 PMCID: PMC6276140 DOI: 10.1186/s12898-018-0209-5] [Citation(s) in RCA: 38] [Impact Index Per Article: 6.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/20/2018] [Accepted: 11/22/2018] [Indexed: 11/10/2022] Open
Abstract
BACKGROUND Phytoplankton species identification and counting is a crucial step of water quality assessment. Especially drinking water reservoirs, bathing and ballast water need to be regularly monitored for harmful species. In times of multiple environmental threats like eutrophication, climate warming and introduction of invasive species more intensive monitoring would be helpful to develop adequate measures. However, traditional methods such as microscopic counting by experts or high throughput flow cytometry based on scattering and fluorescence signals are either too time-consuming or inaccurate for species identification tasks. The combination of high qualitative microscopy with high throughput and latest development in machine learning techniques can overcome this hurdle. RESULTS In this study, image based cytometry was used to collect ~ 47,000 images for brightfield and Chl a fluorescence at 60× magnification for nine common freshwater species of nano- and micro-phytoplankton. A deep neuronal network trained on these images was applied to identify the species and the corresponding life cycle stage during the batch cultivation. The results show the high potential of this approach, where species identity and their respective life cycle stage could be predicted with a high accuracy of 97%. CONCLUSIONS These findings could pave the way for reliable and fast phytoplankton species determination of indicator species as a crucial step in water quality assessment.
Collapse
Affiliation(s)
- Susanne Dunker
- Department of Physiological Diversity, Helmholtz-Centre for Environmental Research-UFZ, Permoserstraße 15, 04318 Leipzig, Germany
- Department of Physiological Diversity, German Centre for Integrative Biodiversity Research-iDiv, Deutscher Platz 5a, 04103 Leipzig, Germany
| | - David Boho
- Software Engineering for Safety-Critical Systems Group, Technische Universität Ilmenau, Ehrenbergstraße 29, 98693 Ilmenau, Germany
| | - Jana Wäldchen
- Department of Biochemical Integration, Max-Planck-Institute for Biogeochemistry, Hans-Knöll-Straße 10, 07745 Jena, Germany
| | - Patrick Mäder
- Software Engineering for Safety-Critical Systems Group, Technische Universität Ilmenau, Ehrenbergstraße 29, 98693 Ilmenau, Germany
| |
Collapse
|
33
|
Beszteri B, Allen C, Almandoz GO, Armand L, Barcena MÁ, Cantzler H, Crosta X, Esper O, Jordan RW, Kauer G, Klaas C, Kloster M, Leventer A, Pike J, Rigual Hernández AS, Wetherbee R. Quantitative comparison of taxa and taxon concepts in the diatom genus Fragilariopsis: a case study on using slide scanning, multiexpert image annotation, and image analysis in taxonomy 1. JOURNAL OF PHYCOLOGY 2018; 54:703-719. [PMID: 30014469 PMCID: PMC6220827 DOI: 10.1111/jpy.12767] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 09/28/2017] [Accepted: 07/07/2018] [Indexed: 06/08/2023]
Abstract
Semiautomated methods for microscopic image acquisition, image analysis, and taxonomic identification have repeatedly received attention in diatom analysis. Less well studied is the question whether and how such methods might prove useful for clarifying the delimitation of species that are difficult to separate for human taxonomists. To try to answer this question, three very similar Fragilariopsis species endemic to the Southern Ocean were targeted in this study: F. obliquecostata, F. ritscheri, and F. sublinearis. A set of 501 extended focus depth specimen images were obtained using a standardized, semiautomated microscopic procedure. Twelve diatomists independently identified these specimen images in order to reconcile taxonomic opinions and agree upon a taxonomic gold standard. Using image analyses, we then extracted morphometric features representing taxonomic characters of the target taxa. The discriminating ability of individual morphometric features was tested visually and statistically, and multivariate classification experiments were performed to test the agreement of the quantitatively defined taxa assignments with expert consensus opinion. Beyond an updated differential diagnosis of the studied taxa, our study also shows that automated imaging and image analysis procedures for diatoms are coming close to reaching a broad applicability for routine use.
Collapse
Affiliation(s)
- Bánk Beszteri
- Section Polar Biological OceanographyAlfred Wegener Institute Helmholtz Centre for Polar and Marine ResearchAm Handelshafen 1227570BremerhavenGermany
| | - Claire Allen
- British Antarctic SurveyHigh Cross, Madingley RoadCambridgeCB3 0ETUK
| | - Gastón O. Almandoz
- División FicologíaFacultad de Ciencias Naturales y MuseoUniversidad Nacional de La PlataPaseo del Bosque s/n (B1900FWA)La PlataArgentina
| | - Leanne Armand
- Research School of Earth SciencesThe Australian National UniversityJaeger Building 4, 142 Mills RoadActonACT2601Australia
| | | | - Hannelore Cantzler
- Section Polar Biological OceanographyAlfred Wegener Institute Helmholtz Centre for Polar and Marine ResearchAm Handelshafen 1227570BremerhavenGermany
| | - Xavier Crosta
- UMR‐CNRS 5805 EPOCUniversité de BordeauxAllée Geoffroy Saint Hilaire33615Pessac CedexFrance
| | - Oliver Esper
- Section Marine GeologyAlfred Wegener Institute Helmholtz Centre for Polar and Marine ResearchAm Handelshafen 1227570BremerhavenGermany
| | - Richard W. Jordan
- Department of Earth & Environmental SciencesFaculty of ScienceYamagata University1‐4‐12 Kojirakawa‐machiYamagata990‐8560Japan
| | - Gerhard Kauer
- BioinformaticsUniversity of Applied SciencesConstantiaplatz 426723EmdenGermany
| | - Christine Klaas
- Section Polar Biological OceanographyAlfred Wegener Institute Helmholtz Centre for Polar and Marine ResearchAm Handelshafen 1227570BremerhavenGermany
| | - Michael Kloster
- Section Polar Biological OceanographyAlfred Wegener Institute Helmholtz Centre for Polar and Marine ResearchAm Handelshafen 1227570BremerhavenGermany
| | - Amy Leventer
- Geology DepartmentColgate UniversityHamiltonNew York13346USA
| | - Jennifer Pike
- School of Earth and Ocean SciencesCardiff UniversityMain Building, Park PlaceCardiffCF10 3ATUK
| | | | | |
Collapse
|
34
|
Ruiz-Santaquiteria J, Espinosa-Aranda JL, Deniz O, Sanchez C, Borrego-Ramos M, Blanco S, Cristobal G, Bueno G. Low-cost oblique illumination: an image quality assessment. JOURNAL OF BIOMEDICAL OPTICS 2018; 23:1-14. [PMID: 29297212 DOI: 10.1117/1.jbo.23.1.016001] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 06/22/2017] [Accepted: 12/04/2017] [Indexed: 06/07/2023]
Abstract
We study the effectiveness of several low-cost oblique illumination filters to improve overall image quality, in comparison with standard bright field imaging. For this purpose, a dataset composed of 3360 diatom images belonging to 21 taxa was acquired. Subjective and objective image quality assessments were done. The subjective evaluation was performed by a group of diatom experts by psychophysical test where resolution, focus, and contrast were assessed. Moreover, some objective nonreference image quality metrics were applied to the same image dataset to complete the study, together with the calculation of several texture features to analyze the effect of these filters in terms of textural properties. Both image quality evaluation methods, subjective and objective, showed better results for images acquired using these illumination filters in comparison with the no filtered image. These promising results confirm that this kind of illumination filters can be a practical way to improve the image quality, thanks to the simple and low cost of the design and manufacturing process.
Collapse
Affiliation(s)
| | | | - Oscar Deniz
- University of Castilla-La Mancha, ETSI Industriales, Visilab, Ciudad Real, Spain
| | - Carlos Sanchez
- Institute of Optics "Daza de Valdés", Spanish National Research Council (CSIC), Madrid, Spain
| | | | - Saul Blanco
- University of León, Institute of Environment, León, Spain
| | - Gabriel Cristobal
- Institute of Optics "Daza de Valdés", Spanish National Research Council (CSIC), Madrid, Spain
| | - Gloria Bueno
- University of Castilla-La Mancha, ETSI Industriales, Visilab, Ciudad Real, Spain
| |
Collapse
|
35
|
Classification of Architectural Heritage Images Using Deep Learning Techniques. APPLIED SCIENCES-BASEL 2017. [DOI: 10.3390/app7100992] [Citation(s) in RCA: 64] [Impact Index Per Article: 9.1] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 12/21/2022]
|
36
|
A survey for the applications of content-based microscopic image analysis in microorganism classification domains. Artif Intell Rev 2017. [DOI: 10.1007/s10462-017-9572-4] [Citation(s) in RCA: 40] [Impact Index Per Article: 5.7] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/16/2023]
|
37
|
Automated Diatom Classification (Part A): Handcrafted Feature Approaches. APPLIED SCIENCES-BASEL 2017. [DOI: 10.3390/app7080753] [Citation(s) in RCA: 20] [Impact Index Per Article: 2.9] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/16/2022]
|